[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:09 dist-test-slave-8n49 krb5kdc[19211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:09 dist-test-slave-8n49 krb5kdc[19211](info): set up 2 sockets
Aug 07 04:09:09 dist-test-slave-8n49 krb5kdc[19211](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 04:09:10.637146 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:10 dist-test-slave-8n49 krb5kdc[19211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:10.846671 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33313
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:33313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:11.477016 19225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:11.477491 19225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:11.477705 19225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:11.487458 19225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:11.487877 19225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:11.488055 19225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:11.488268 19225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:11.488415 19225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:11.502524 19225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:33313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:11.504345 19225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:11.505977 19225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:11.522948 19231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:11.523136 19230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:11.523924 19233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:11.524948 19225 server_base.cc:1034] running on GCE node
I20260807 04:09:11.526028 19225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:11.528084 19225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:11.529290 19225 hybrid_clock.cc:648] HybridClock initialized: now 1786075751529221 us; error 60 us; skew 500 ppm
I20260807 04:09:11.529659 19225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:11 dist-test-slave-8n49 krb5kdc[19211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075751, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:11.539113 19225 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:11.541343 19225 webserver.cc:460] Webserver started at http://127.18.193.62:40837/ using document root <none> and password file <none>
I20260807 04:09:11.542337 19225 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:11.542505 19225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:11.542830 19225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:11.547015 19225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "ded836744ec74621965de5da7ea2366f"
format_stamp: "Formatted at 2026-08-07 04:09:11 on dist-test-slave-8n49"
server_key: "1362f7a7e0d4b57f3506779c0e8c2e69"
server_key_iv: "69e3dc0f7edc14e01f17392c40c2b65b"
server_key_version: "encryptionkey@0"
I20260807 04:09:11.547884 19225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "ded836744ec74621965de5da7ea2366f"
format_stamp: "Formatted at 2026-08-07 04:09:11 on dist-test-slave-8n49"
server_key: "1362f7a7e0d4b57f3506779c0e8c2e69"
server_key_iv: "69e3dc0f7edc14e01f17392c40c2b65b"
server_key_version: "encryptionkey@0"
I20260807 04:09:11.555240 19225 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.006s
I20260807 04:09:11.560297 19240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:11.562073 19225 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 04:09:11.562338 19225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "ded836744ec74621965de5da7ea2366f"
format_stamp: "Formatted at 2026-08-07 04:09:11 on dist-test-slave-8n49"
server_key: "1362f7a7e0d4b57f3506779c0e8c2e69"
server_key_iv: "69e3dc0f7edc14e01f17392c40c2b65b"
server_key_version: "encryptionkey@0"
I20260807 04:09:11.562667 19225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:11.599051 19225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:11.605204 19225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:11.605701 19225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:11.645944 19225 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:33313
I20260807 04:09:11.646059 19294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:33313 every 8 connection(s)
I20260807 04:09:11.647848 19225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:11.653177 19295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:11.654102 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 19225
I20260807 04:09:11.654647 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:11.658061 19204 external_mini_cluster.cc:1428] Setting key 3948dd8dcafe9f551f2c5db624a60443
I20260807 04:09:11.670559 19295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f: Bootstrap starting.
I20260807 04:09:11.674378 19295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:11.675773 19295 log.cc:824] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:11.680472 19295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f: No bootstrap required, opened a new log
I20260807 04:09:11.692762 19295 raft_consensus.cc:348] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded836744ec74621965de5da7ea2366f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33313 } }
I20260807 04:09:11.693197 19295 raft_consensus.cc:374] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:11.693370 19295 raft_consensus.cc:729] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded836744ec74621965de5da7ea2366f, State: Initialized, Role: FOLLOWER
I20260807 04:09:11.694123 19295 consensus_queue.cc:260] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded836744ec74621965de5da7ea2366f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33313 } }
I20260807 04:09:11.694435 19295 raft_consensus.cc:388] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:11.694594 19295 raft_consensus.cc:482] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:11.694805 19295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 0 FOLLOWER]: Advancing to term 1
Aug 07 04:09:11 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:11.698175 19295 raft_consensus.cc:504] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded836744ec74621965de5da7ea2366f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33313 } }
I20260807 04:09:11.699051 19295 leader_election.cc:302] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ded836744ec74621965de5da7ea2366f; no voters:
I20260807 04:09:11.700418 19295 leader_election.cc:288] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:11.700708 19300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:11.702421 19300 raft_consensus.cc:686] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [term 1 LEADER]: Becoming Leader. State: Replica: ded836744ec74621965de5da7ea2366f, State: Running, Role: LEADER
I20260807 04:09:11.703675 19295 sys_catalog.cc:564] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:11.703469 19300 consensus_queue.cc:237] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded836744ec74621965de5da7ea2366f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33313 } }
I20260807 04:09:11.717195 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:11.664314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.18.193.62:33313)
0807 04:09:11.665614 (+ 1300us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:11.665632 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:11.667631 (+ 1999us) server_negotiation.cc:407] Connection header received
0807 04:09:11.680323 (+ 12692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:11.680347 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:11.680969 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:11.681556 (+ 587us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:11.685094 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:11.687927 (+ 2833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:11.690209 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:11.691608 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:11.691822 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:11.698478 (+ 6656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:11.698492 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:11.698511 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:11.698562 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:11.705220 (+ 6658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:11.706755 (+ 1535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:11.706765 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:11.706779 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:11.707136 (+ 357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:11.708092 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:11.708100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:11.708106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:11.709418 (+ 1312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:11.709991 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:11.713897 (+ 3906us) server_negotiation.cc:299] Negotiation successful
0807 04:09:11.714510 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1040,"thread_start_us":673,"threads_started":1}
I20260807 04:09:11.718943 19302 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded836744ec74621965de5da7ea2366f. Latest consensus state: current_term: 1 leader_uuid: "ded836744ec74621965de5da7ea2366f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded836744ec74621965de5da7ea2366f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33313 } } }
I20260807 04:09:11.719820 19302 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:11.720412 19301 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ded836744ec74621965de5da7ea2366f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded836744ec74621965de5da7ea2366f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33313 } } }
I20260807 04:09:11.721302 19301 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f [sys.catalog]: This master's current role is: LEADER
W20260807 04:09:11.728605 19316 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:09:11.728842 19316 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:09:11.729260 19314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:11.733292 19314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:11.746598 19314 catalog_manager.cc:1347] Generated new cluster ID: 46a0748fbbfc4736baa86ab232daaada
I20260807 04:09:11.746796 19314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:11.764163 19314 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:11.766855 19314 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:11.780270 19314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ded836744ec74621965de5da7ea2366f: Generated new TSK 0
I20260807 04:09:11.781543 19314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:11.837445 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33313
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:12.400302 19322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:12.400754 19322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:12.400985 19322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:12.410806 19322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:12.410959 19322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:12.411201 19322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:12.423346 19322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33313
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:12.424929 19322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:12.426584 19322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:12.444646 19327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:12.444736 19328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:12.445094 19330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:12.446390 19322 server_base.cc:1034] running on GCE node
I20260807 04:09:12.447131 19322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:12.448315 19322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:12.449587 19322 hybrid_clock.cc:648] HybridClock initialized: now 1786075752449476 us; error 82 us; skew 500 ppm
I20260807 04:09:12.449949 19322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:12 dist-test-slave-8n49 krb5kdc[19211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:12.453673 19322 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:12.455513 19322 webserver.cc:460] Webserver started at http://127.18.193.1:37433/ using document root <none> and password file <none>
I20260807 04:09:12.456329 19322 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:12.456547 19322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:12.457062 19322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:12.460749 19322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "f7b8066712fd446ab930dba5b4002d7b"
format_stamp: "Formatted at 2026-08-07 04:09:12 on dist-test-slave-8n49"
server_key: "f5ab6016d6885124fded77f69e46f39b"
server_key_iv: "0288f6ccf03b829c91e9a504e59f2b7b"
server_key_version: "encryptionkey@0"
I20260807 04:09:12.461661 19322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "f7b8066712fd446ab930dba5b4002d7b"
format_stamp: "Formatted at 2026-08-07 04:09:12 on dist-test-slave-8n49"
server_key: "f5ab6016d6885124fded77f69e46f39b"
server_key_iv: "0288f6ccf03b829c91e9a504e59f2b7b"
server_key_version: "encryptionkey@0"
I20260807 04:09:12.468407 19322 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 04:09:12.473605 19338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:12.475162 19322 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:09:12.475409 19322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "f7b8066712fd446ab930dba5b4002d7b"
format_stamp: "Formatted at 2026-08-07 04:09:12 on dist-test-slave-8n49"
server_key: "f5ab6016d6885124fded77f69e46f39b"
server_key_iv: "0288f6ccf03b829c91e9a504e59f2b7b"
server_key_version: "encryptionkey@0"
I20260807 04:09:12.475714 19322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:12.494253 19322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:12.498847 19322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:12.499228 19322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:12.500614 19322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:12.502317 19322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:12.502466 19322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:12.502673 19322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:12.502796 19322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:12.571305 19322 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:41197
I20260807 04:09:12.571463 19455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:41197 every 8 connection(s)
I20260807 04:09:12.572966 19322 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:12.575121 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 19322
I20260807 04:09:12.575459 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:12.576217 19204 external_mini_cluster.cc:1428] Setting key df814a3cfca27b0ed7c75ddcb46cd9b1
Aug 07 04:09:12 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
I20260807 04:09:12.605775 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:12.577213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:42881 (local address 127.18.193.62:33313)
0807 04:09:12.578107 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:12.578113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:12.579776 (+ 1663us) server_negotiation.cc:407] Connection header received
0807 04:09:12.583689 (+ 3913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:12.583694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:12.583767 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:12.583890 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:12.586740 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:12.588087 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:12.591241 (+ 3154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:12.591778 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:12.591883 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:12.598076 (+ 6193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:12.598081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:12.598085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:12.598123 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:12.601670 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:12.602513 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:12.602517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:12.602526 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:12.602724 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:12.603217 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:12.603221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:12.603223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:12.603692 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:12.603878 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:12.604765 (+ 887us) server_negotiation.cc:299] Negotiation successful
0807 04:09:12.605087 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":511,"threads_started":1}
I20260807 04:09:12.606560 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:12.578090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33313 (local address 127.18.193.1:42881)
0807 04:09:12.579599 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:12.579653 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:12.583324 (+ 3671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:12.584239 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:12.584254 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:12.585328 (+ 1074us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:12.586526 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:12.586550 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:12.588254 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:12.588263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:12.591072 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:12.591092 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:12.592126 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:12.592137 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:12.592601 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:12.593550 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:12.593611 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:12.597765 (+ 4154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:12.601824 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:12.601841 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:12.601856 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:12.602383 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:12.602844 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:12.602847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:12.602849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:12.603109 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:12.603872 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:12.603879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:12.604305 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 04:09:12.604596 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 04:09:12.605145 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":700,"threads_started":1}
I20260807 04:09:12.608778 19456 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33313
I20260807 04:09:12.609179 19456 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:12.610337 19456 heartbeater.cc:511] Master 127.18.193.62:33313 requested a full tablet report, sending...
I20260807 04:09:12.613238 19257 ts_manager.cc:194] Registered new tserver with Master: f7b8066712fd446ab930dba5b4002d7b (127.18.193.1:41197)
I20260807 04:09:12.615821 19257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:42881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:12.632146 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33313
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:13.187747 19462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:13.188253 19462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:13.188521 19462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:13.197984 19462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:13.198236 19462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:13.198472 19462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:13.210879 19462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33313
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:13.212394 19462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:13.214200 19462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:13.231958 19467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:13.232592 19468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:13.233053 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:13.234331 19462 server_base.cc:1034] running on GCE node
I20260807 04:09:13.235121 19462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:13.236209 19462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:13.237447 19462 hybrid_clock.cc:648] HybridClock initialized: now 1786075753237350 us; error 47 us; skew 500 ppm
I20260807 04:09:13.237871 19462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:13 dist-test-slave-8n49 krb5kdc[19211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075753, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:13.242272 19462 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:13.244129 19462 webserver.cc:460] Webserver started at http://127.18.193.2:41145/ using document root <none> and password file <none>
I20260807 04:09:13.245012 19462 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:13.245225 19462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:13.245623 19462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:13.249372 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "9da96c3744bb4abda15ed094ab778a06"
format_stamp: "Formatted at 2026-08-07 04:09:13 on dist-test-slave-8n49"
server_key: "290541ec7a00f82ab23956dcce983279"
server_key_iv: "46560002600f68d47000a8947e1d36e5"
server_key_version: "encryptionkey@0"
I20260807 04:09:13.250317 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "9da96c3744bb4abda15ed094ab778a06"
format_stamp: "Formatted at 2026-08-07 04:09:13 on dist-test-slave-8n49"
server_key: "290541ec7a00f82ab23956dcce983279"
server_key_iv: "46560002600f68d47000a8947e1d36e5"
server_key_version: "encryptionkey@0"
I20260807 04:09:13.257256 19462 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 04:09:13.261979 19478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:13.263540 19462 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 04:09:13.263809 19462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "9da96c3744bb4abda15ed094ab778a06"
format_stamp: "Formatted at 2026-08-07 04:09:13 on dist-test-slave-8n49"
server_key: "290541ec7a00f82ab23956dcce983279"
server_key_iv: "46560002600f68d47000a8947e1d36e5"
server_key_version: "encryptionkey@0"
I20260807 04:09:13.264103 19462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:13.288169 19462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:13.293520 19462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:13.293951 19462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:13.295305 19462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:13.297271 19462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:13.297444 19462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:13.297727 19462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:13.297873 19462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:13.374871 19462 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:46557
I20260807 04:09:13.375064 19595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:46557 every 8 connection(s)
I20260807 04:09:13.377161 19462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:13.387372 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 19462
I20260807 04:09:13.387657 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:13.388134 19204 external_mini_cluster.cc:1428] Setting key 032f6bc6502ad20098137cf6e4b21853
Aug 07 04:09:13 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075753, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:13.405509 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:13.381052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:36587 (local address 127.18.193.62:33313)
0807 04:09:13.381898 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:13.381902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:13.383008 (+ 1106us) server_negotiation.cc:407] Connection header received
0807 04:09:13.385241 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:13.385245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:13.385315 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:13.385426 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:13.387213 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:13.388289 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:13.390601 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:13.391226 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:13.391322 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:13.397278 (+ 5956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:13.397284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:13.397287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:13.397319 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:13.401030 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:13.402169 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:13.402173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:13.402175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:13.402364 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:13.403046 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:13.403050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:13.403052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:13.403454 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:13.403580 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:13.404731 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0807 04:09:13.405008 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":590,"threads_started":1}
I20260807 04:09:13.407207 19599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:13.381592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33313 (local address 127.18.193.2:36587)
0807 04:09:13.382801 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:13.382848 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:13.384969 (+ 2121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:13.385638 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:13.385647 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:13.386335 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:13.386976 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:13.387014 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:13.388442 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:13.388448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:13.390429 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:13.390442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:13.391520 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:13.391530 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:13.392041 (+ 511us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:13.393086 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:13.393121 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:13.396956 (+ 3835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:13.401185 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:13.401198 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:13.401219 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:13.402020 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:13.402493 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:13.402501 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:13.402508 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:13.402875 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:13.403606 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:13.403616 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:13.404210 (+ 594us) client_negotiation.cc:769] Sending connection context
0807 04:09:13.404576 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 04:09:13.405350 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"thread_start_us":558,"threads_started":1}
I20260807 04:09:13.409126 19596 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33313
I20260807 04:09:13.409580 19596 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:13.410667 19596 heartbeater.cc:511] Master 127.18.193.62:33313 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
I20260807 04:09:13.412523 19257 ts_manager.cc:194] Registered new tserver with Master: 9da96c3744bb4abda15ed094ab778a06 (127.18.193.2:46557)
I20260807 04:09:13.413614 19257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:36587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:13.440886 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33313
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:09:13.619437 19456 heartbeater.cc:503] Master 127.18.193.62:33313 was elected leader, sending a full tablet report...
W20260807 04:09:13.990716 19602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:13.991084 19602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:13.991321 19602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:14.002161 19602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:14.002379 19602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:14.002552 19602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:14.016947 19602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33313
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:14.018548 19602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:14.020232 19602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:14.043237 19610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:14.043429 19607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:14.043524 19608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:14.045218 19602 server_base.cc:1034] running on GCE node
I20260807 04:09:14.046130 19602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:14.047602 19602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:14.048941 19602 hybrid_clock.cc:648] HybridClock initialized: now 1786075754048810 us; error 64 us; skew 500 ppm
I20260807 04:09:14.049357 19602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:14.054378 19602 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:14.056545 19602 webserver.cc:460] Webserver started at http://127.18.193.3:46847/ using document root <none> and password file <none>
I20260807 04:09:14.057458 19602 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:14.057716 19602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:14.058344 19602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:14.062270 19602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "7c43aa5ae1d5461b83435afae1ef158f"
format_stamp: "Formatted at 2026-08-07 04:09:14 on dist-test-slave-8n49"
server_key: "59d9a9c46a5c06e351d2b57290153fca"
server_key_iv: "34636973153676d891599dfb8a32574b"
server_key_version: "encryptionkey@0"
I20260807 04:09:14.063345 19602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "7c43aa5ae1d5461b83435afae1ef158f"
format_stamp: "Formatted at 2026-08-07 04:09:14 on dist-test-slave-8n49"
server_key: "59d9a9c46a5c06e351d2b57290153fca"
server_key_iv: "34636973153676d891599dfb8a32574b"
server_key_version: "encryptionkey@0"
I20260807 04:09:14.070232 19602 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260807 04:09:14.075814 19617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:14.077498 19602 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260807 04:09:14.077723 19602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "7c43aa5ae1d5461b83435afae1ef158f"
format_stamp: "Formatted at 2026-08-07 04:09:14 on dist-test-slave-8n49"
server_key: "59d9a9c46a5c06e351d2b57290153fca"
server_key_iv: "34636973153676d891599dfb8a32574b"
server_key_version: "encryptionkey@0"
I20260807 04:09:14.078011 19602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:14.100538 19602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:14.107228 19602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:14.107744 19602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:14.109498 19602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:14.111719 19602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:14.111850 19602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:14.112157 19602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:14.112283 19602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:14.187781 19602 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:45565
I20260807 04:09:14.188022 19736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:45565 every 8 connection(s)
I20260807 04:09:14.190114 19602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:14.195480 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 19602
I20260807 04:09:14.195756 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:14.196602 19204 external_mini_cluster.cc:1428] Setting key 73f383ee40762cc97bf89f58ba3f15e0
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:14.220042 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:14.193346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:42247 (local address 127.18.193.62:33313)
0807 04:09:14.194170 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:14.194175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:14.195504 (+ 1329us) server_negotiation.cc:407] Connection header received
0807 04:09:14.198981 (+ 3477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:14.198987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:14.199101 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:14.199243 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:14.201712 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.202746 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.204876 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.205811 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.206020 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:14.211548 (+ 5528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:14.211554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:14.211557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:14.211601 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:14.215925 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.216964 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.216969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.216971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.217171 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.217781 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.217785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.217787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.218130 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:14.218267 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:14.219371 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0807 04:09:14.219542 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":526,"threads_started":1}
I20260807 04:09:14.221832 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:14.193894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33313 (local address 127.18.193.3:42247)
0807 04:09:14.195278 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:14.195327 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:14.198667 (+ 3340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:14.199562 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:14.199574 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:14.200404 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:14.201497 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:14.201537 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.202930 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.202934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:14.204736 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:14.204750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.206195 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.206202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:14.206538 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:14.207457 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:14.207484 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:14.211199 (+ 3715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:14.216114 (+ 4915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:14.216122 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:14.216138 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:14.216840 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:14.217275 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:14.217279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:14.217281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:14.217655 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:14.218261 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:14.218273 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:14.218847 (+ 574us) client_negotiation.cc:769] Sending connection context
0807 04:09:14.219287 (+ 440us) client_negotiation.cc:240] Negotiation successful
0807 04:09:14.219969 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"thread_start_us":619,"threads_started":1}
I20260807 04:09:14.223537 19737 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33313
I20260807 04:09:14.223871 19737 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:14.224931 19737 heartbeater.cc:511] Master 127.18.193.62:33313 requested a full tablet report, sending...
I20260807 04:09:14.226848 19257 ts_manager.cc:194] Registered new tserver with Master: 7c43aa5ae1d5461b83435afae1ef158f (127.18.193.3:45565)
I20260807 04:09:14.227906 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:09:14.228256 19257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:42247
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:14.342970 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:14.320407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57718 (local address 127.18.193.1:41197)
0807 04:09:14.321335 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:14.321361 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:14.322055 (+ 694us) server_negotiation.cc:407] Connection header received
0807 04:09:14.322334 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:14.322343 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:14.322594 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:14.322770 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:14.326020 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.327173 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.328872 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.329564 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.329730 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:14.335116 (+ 5386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:14.335131 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:14.335157 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:14.335229 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:14.339679 (+ 4450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.340558 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.340590 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.340593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.340836 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.341259 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.341263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.341265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.341723 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:14.342099 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:14.342246 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 04:09:14.342502 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":475,"threads_started":1}
W20260807 04:09:14.344547 19369 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57718
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260807 04:09:14.417234 19596 heartbeater.cc:503] Master 127.18.193.62:33313 was elected leader, sending a full tablet report...
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:14.438026 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:14.420342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.18.193.1:41197)
0807 04:09:14.420620 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:14.420628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:14.420907 (+ 279us) server_negotiation.cc:407] Connection header received
0807 04:09:14.421135 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:14.421140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:14.421206 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:14.421325 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:14.423103 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.423817 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.425389 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.425819 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.425958 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:14.430976 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:14.430981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:14.430984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:14.431034 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:14.434490 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.435426 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.435430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.435433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.435660 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.436155 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.436160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.436162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.436630 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:14.436937 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:14.437249 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 04:09:14.437498 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 04:09:14.440125 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 19322
I20260807 04:09:14.456440 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:14.441324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.18.193.1:41197)
0807 04:09:14.441509 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:14.441519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:14.441842 (+ 323us) server_negotiation.cc:407] Connection header received
0807 04:09:14.441972 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:14.441976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:14.442036 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:14.442121 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:14.443667 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.444432 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.446222 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.446884 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.446991 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:14.449863 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:14.449868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:14.449871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:14.449900 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:14.453255 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.453935 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.453938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.453941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.454137 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.454513 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.454516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.454518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.454838 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:14.455077 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:14.455857 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 04:09:14.456018 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:09:14.457373 19451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:14.551180 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 19322
I20260807 04:09:14.570477 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 19462
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:09:14.586797 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:14.571448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.18.193.2:46557)
0807 04:09:14.572103 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:14.572110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:14.572135 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:09:14.572196 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:14.572202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:14.572425 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:14.572631 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:14.574033 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.575033 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.576234 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.576931 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.577029 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:14.581164 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:14.581174 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:14.581188 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:14.581220 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:14.584179 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.584877 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.584884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.584886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.585004 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.585334 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.585337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.585339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.585670 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:14.585890 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:14.586213 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 04:09:14.586398 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":359,"threads_started":1}
I20260807 04:09:14.588100 19591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:14.674902 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 19462
I20260807 04:09:14.695683 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 19602
Aug 07 04:09:14 dist-test-slave-8n49 krb5kdc[19211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:09:14.713258 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:14.696859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.18.193.3:45565)
0807 04:09:14.697473 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:14.697481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:14.697514 (+ 33us) server_negotiation.cc:407] Connection header received
0807 04:09:14.697582 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:14.697589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:14.697804 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:14.697938 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:14.699228 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.700257 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.701694 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:14.702268 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:14.702376 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:14.706669 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:14.706681 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:14.706697 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:14.706737 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:14.710000 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.710805 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.710813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.710815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.710966 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:14.711500 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:14.711504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:14.711506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:14.711917 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:14.712183 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:14.712671 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 04:09:14.712905 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":331,"threads_started":1}
I20260807 04:09:14.714691 19731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:14.801000 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 19602
I20260807 04:09:14.819878 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 19225
I20260807 04:09:14.821431 19289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:14.930331 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 19225
2026-08-07T04:09:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5459 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:15 dist-test-slave-8n49 krb5kdc[19760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:15 dist-test-slave-8n49 krb5kdc[19760](info): set up 2 sockets
Aug 07 04:09:15 dist-test-slave-8n49 krb5kdc[19760](info): commencing operation
krb5kdc: starting...
W20260807 04:09:16.931021 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.955s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:17 dist-test-slave-8n49 krb5kdc[19760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:17.144186 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38521
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:38521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:17.476320 19773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:17.476725 19773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:17.476895 19773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:17.484258 19773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:17.484369 19773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:17.484452 19773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:17.484546 19773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:17.484654 19773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:17.494804 19773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:38521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:17.496163 19773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:17.497509 19773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:17.510134 19783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:17.510221 19780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:17.510185 19778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:17.511363 19773 server_base.cc:1034] running on GCE node
I20260807 04:09:17.512151 19773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:17.513650 19773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:17.514906 19773 hybrid_clock.cc:648] HybridClock initialized: now 1786075757514846 us; error 37 us; skew 500 ppm
I20260807 04:09:17.515259 19773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:17 dist-test-slave-8n49 krb5kdc[19760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075757, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:17.524104 19773 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:17.525883 19773 webserver.cc:460] Webserver started at http://127.18.193.62:45761/ using document root <none> and password file <none>
I20260807 04:09:17.526639 19773 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:17.526778 19773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:17.527132 19773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:17.530337 19773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "e3f557834ac144b5b477a75c04ee8659"
format_stamp: "Formatted at 2026-08-07 04:09:17 on dist-test-slave-8n49"
server_key: "a5daea74138de8828abe463ceac951e0"
server_key_iv: "be58487f89de046c4f775c544fbffc7d"
server_key_version: "encryptionkey@0"
I20260807 04:09:17.531173 19773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "e3f557834ac144b5b477a75c04ee8659"
format_stamp: "Formatted at 2026-08-07 04:09:17 on dist-test-slave-8n49"
server_key: "a5daea74138de8828abe463ceac951e0"
server_key_iv: "be58487f89de046c4f775c544fbffc7d"
server_key_version: "encryptionkey@0"
I20260807 04:09:17.536644 19773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:09:17.540158 19790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:17.541574 19773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:17.541760 19773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "e3f557834ac144b5b477a75c04ee8659"
format_stamp: "Formatted at 2026-08-07 04:09:17 on dist-test-slave-8n49"
server_key: "a5daea74138de8828abe463ceac951e0"
server_key_iv: "be58487f89de046c4f775c544fbffc7d"
server_key_version: "encryptionkey@0"
I20260807 04:09:17.541967 19773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:17.573154 19773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:17.577587 19773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:17.577905 19773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:17.597267 19773 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:38521
I20260807 04:09:17.597287 19843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:38521 every 8 connection(s)
I20260807 04:09:17.598726 19773 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:17.602614 19844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:17.604025 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 19773
I20260807 04:09:17.604286 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:17.605173 19204 external_mini_cluster.cc:1428] Setting key 8ff0c05e39a7c2a8a0946c16c0e37bca
I20260807 04:09:17.618465 19844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659: Bootstrap starting.
Aug 07 04:09:17 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:17.622473 19844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:17.623972 19844 log.cc:824] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:17.627494 19844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659: No bootstrap required, opened a new log
I20260807 04:09:17.630497 19847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:17.607317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.18.193.62:38521)
0807 04:09:17.608222 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:17.608265 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:17.608325 (+ 60us) server_negotiation.cc:407] Connection header received
0807 04:09:17.610240 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:17.610267 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:17.610729 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:17.611083 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:17.612979 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:17.614261 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:17.616066 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:17.616816 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:17.616957 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:17.621811 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:17.621821 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:17.621840 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:17.621894 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:17.625783 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:17.626515 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:17.626525 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:17.626533 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:17.626763 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:17.627133 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:17.627138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:17.627141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:17.627844 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:17.628171 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:17.628638 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 04:09:17.629078 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":399,"threads_started":1}
I20260807 04:09:17.636757 19844 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f557834ac144b5b477a75c04ee8659" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38521 } }
I20260807 04:09:17.637130 19844 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:17.637217 19844 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f557834ac144b5b477a75c04ee8659, State: Initialized, Role: FOLLOWER
I20260807 04:09:17.637744 19844 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f557834ac144b5b477a75c04ee8659" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38521 } }
I20260807 04:09:17.637951 19844 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:17.638051 19844 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:17.638175 19844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:17.640309 19844 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f557834ac144b5b477a75c04ee8659" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38521 } }
I20260807 04:09:17.640841 19844 leader_election.cc:302] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3f557834ac144b5b477a75c04ee8659; no voters:
I20260807 04:09:17.641425 19844 leader_election.cc:288] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:17.641603 19850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:17.642357 19850 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [term 1 LEADER]: Becoming Leader. State: Replica: e3f557834ac144b5b477a75c04ee8659, State: Running, Role: LEADER
I20260807 04:09:17.642891 19850 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f557834ac144b5b477a75c04ee8659" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38521 } }
I20260807 04:09:17.643581 19844 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:17.645879 19851 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3f557834ac144b5b477a75c04ee8659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f557834ac144b5b477a75c04ee8659" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38521 } } }
I20260807 04:09:17.646366 19851 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:17.648101 19856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:17.648331 19852 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f557834ac144b5b477a75c04ee8659. Latest consensus state: current_term: 1 leader_uuid: "e3f557834ac144b5b477a75c04ee8659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f557834ac144b5b477a75c04ee8659" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38521 } } }
I20260807 04:09:17.648819 19852 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:17.653878 19856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:17.663093 19856 catalog_manager.cc:1347] Generated new cluster ID: 8bb4cff2afa3475fbefa263f75b19013
I20260807 04:09:17.663221 19856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:17.683588 19856 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:17.685014 19856 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:17.694368 19856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3f557834ac144b5b477a75c04ee8659: Generated new TSK 0
I20260807 04:09:17.695220 19856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:17.748293 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38521
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:18.087617 19872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:18.087915 19872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:18.088020 19872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:18.095366 19872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:18.095504 19872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:18.095626 19872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:18.106637 19872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:18.108021 19872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:18.109481 19872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:18.123499 19877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:18.123545 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:18.123512 19880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:18.124377 19872 server_base.cc:1034] running on GCE node
I20260807 04:09:18.125041 19872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:18.126197 19872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:18.127460 19872 hybrid_clock.cc:648] HybridClock initialized: now 1786075758127391 us; error 56 us; skew 500 ppm
I20260807 04:09:18.127810 19872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:18 dist-test-slave-8n49 krb5kdc[19760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:18.131384 19872 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:18.133036 19872 webserver.cc:460] Webserver started at http://127.18.193.1:46503/ using document root <none> and password file <none>
I20260807 04:09:18.133744 19872 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:18.133848 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:18.134127 19872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:18.137243 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "78815cef9e6f4d8cb410787bf93430a3"
format_stamp: "Formatted at 2026-08-07 04:09:18 on dist-test-slave-8n49"
server_key: "c6395b984caff4a7bcf70485cd360d17"
server_key_iv: "52e91f95b872441a252f2d97cde61ac4"
server_key_version: "encryptionkey@0"
I20260807 04:09:18.138018 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "78815cef9e6f4d8cb410787bf93430a3"
format_stamp: "Formatted at 2026-08-07 04:09:18 on dist-test-slave-8n49"
server_key: "c6395b984caff4a7bcf70485cd360d17"
server_key_iv: "52e91f95b872441a252f2d97cde61ac4"
server_key_version: "encryptionkey@0"
I20260807 04:09:18.143450 19872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:09:18.147634 19888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:18.149156 19872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:18.149292 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "78815cef9e6f4d8cb410787bf93430a3"
format_stamp: "Formatted at 2026-08-07 04:09:18 on dist-test-slave-8n49"
server_key: "c6395b984caff4a7bcf70485cd360d17"
server_key_iv: "52e91f95b872441a252f2d97cde61ac4"
server_key_version: "encryptionkey@0"
I20260807 04:09:18.149473 19872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:18.168465 19872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:18.173143 19872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:18.173449 19872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:18.174430 19872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:18.175799 19872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:18.175874 19872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:18.176028 19872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:18.176077 19872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:18.212011 19872 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:39665
I20260807 04:09:18.212047 20004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:39665 every 8 connection(s)
I20260807 04:09:18.213595 19872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:18.218561 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 19872
I20260807 04:09:18.218794 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:18.219424 19204 external_mini_cluster.cc:1428] Setting key ec1371b26685de8d96dd2eafe71c273d
Aug 07 04:09:18 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:18.237170 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:18.216351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:50677 (local address 127.18.193.62:38521)
0807 04:09:18.216893 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:18.216897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:18.217754 (+ 857us) server_negotiation.cc:407] Connection header received
0807 04:09:18.219749 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:18.219755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:18.219849 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:18.219972 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:18.221536 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.222260 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.224051 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.224797 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.224931 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:18.229074 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:18.229092 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:18.229100 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:18.229147 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:18.232900 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:18.234150 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:18.234160 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:18.234167 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:18.234345 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:18.234808 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:18.234816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:18.234821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:18.235259 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:18.235431 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:18.236331 (+ 900us) server_negotiation.cc:299] Negotiation successful
0807 04:09:18.236657 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":330,"threads_started":1}
I20260807 04:09:18.237808 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:18.216878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38521 (local address 127.18.193.1:50677)
0807 04:09:18.217582 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:18.217632 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:18.219440 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:18.220226 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:18.220235 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:18.220803 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:18.221368 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:18.221386 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.222421 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.222430 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:18.223907 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:18.223924 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.225046 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.225053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:18.225306 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:18.226031 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:18.226059 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:18.228781 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:18.233098 (+ 4317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:18.233110 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:18.233130 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:18.233887 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:18.234467 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:18.234470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:18.234473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:18.234673 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:18.235439 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:18.235447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:18.235878 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 04:09:18.236143 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 04:09:18.236645 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":279,"threads_started":1}
I20260807 04:09:18.239523 20005 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38521
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
I20260807 04:09:18.239879 20005 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:18.240751 20005 heartbeater.cc:511] Master 127.18.193.62:38521 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
I20260807 04:09:18.243000 19808 ts_manager.cc:194] Registered new tserver with Master: 78815cef9e6f4d8cb410787bf93430a3 (127.18.193.1:39665)
I20260807 04:09:18.245368 19808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:50677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:18.269387 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38521
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:18.614796 20011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:18.615108 20011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:18.615314 20011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:18.622979 20011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:18.623100 20011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:18.623294 20011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:18.633397 20011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:18.634800 20011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:18.636209 20011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:18.650648 20016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:18.650687 20019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:18.650856 20017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:18.651932 20011 server_base.cc:1034] running on GCE node
I20260807 04:09:18.652813 20011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:18.653923 20011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:18.655117 20011 hybrid_clock.cc:648] HybridClock initialized: now 1786075758655062 us; error 43 us; skew 500 ppm
I20260807 04:09:18.655520 20011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:18 dist-test-slave-8n49 krb5kdc[19760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:18.658986 20011 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:18.660743 20011 webserver.cc:460] Webserver started at http://127.18.193.2:45063/ using document root <none> and password file <none>
I20260807 04:09:18.661515 20011 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:18.661675 20011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:18.662009 20011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:18.665349 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "21d80a7207ff40538f101b009230eadd"
format_stamp: "Formatted at 2026-08-07 04:09:18 on dist-test-slave-8n49"
server_key: "4915f3c97a8f20ca9e5e747587dd306e"
server_key_iv: "bc133c3c69bd00fda060ddae7a3e6236"
server_key_version: "encryptionkey@0"
I20260807 04:09:18.666241 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "21d80a7207ff40538f101b009230eadd"
format_stamp: "Formatted at 2026-08-07 04:09:18 on dist-test-slave-8n49"
server_key: "4915f3c97a8f20ca9e5e747587dd306e"
server_key_iv: "bc133c3c69bd00fda060ddae7a3e6236"
server_key_version: "encryptionkey@0"
I20260807 04:09:18.671643 20011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:09:18.675293 20026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:18.676728 20011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:09:18.676909 20011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "21d80a7207ff40538f101b009230eadd"
format_stamp: "Formatted at 2026-08-07 04:09:18 on dist-test-slave-8n49"
server_key: "4915f3c97a8f20ca9e5e747587dd306e"
server_key_iv: "bc133c3c69bd00fda060ddae7a3e6236"
server_key_version: "encryptionkey@0"
I20260807 04:09:18.677141 20011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:18.713425 20011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:18.717964 20011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:18.718324 20011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:18.719475 20011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:18.721139 20011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:18.721225 20011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:18.721330 20011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:18.721381 20011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:18.755815 20011 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:45875
I20260807 04:09:18.755867 20142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:45875 every 8 connection(s)
I20260807 04:09:18.757189 20011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:18.760162 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 20011
I20260807 04:09:18.760377 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:18.760912 19204 external_mini_cluster.cc:1428] Setting key 633fd9e350a50ae0b4745e5fadf71a44
Aug 07 04:09:18 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:18.779775 20145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:18.759691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:36185 (local address 127.18.193.62:38521)
0807 04:09:18.760234 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:18.760241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:18.761404 (+ 1163us) server_negotiation.cc:407] Connection header received
0807 04:09:18.763298 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:18.763304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:18.763380 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:18.763511 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:18.765053 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.766264 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.768613 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.769274 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.769429 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:18.773481 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:18.773486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:18.773490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:18.773523 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:18.776216 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:18.777304 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:18.777307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:18.777310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:18.777442 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:18.777971 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:18.777974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:18.777977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:18.778261 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:18.778371 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:18.779305 (+ 934us) server_negotiation.cc:299] Negotiation successful
0807 04:09:18.779476 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":258,"threads_started":1}
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
I20260807 04:09:18.781371 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:18.760304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38521 (local address 127.18.193.2:36185)
0807 04:09:18.761197 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:18.761258 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:18.763012 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:18.763721 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:18.763729 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:18.764227 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:18.764874 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:18.764893 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.766442 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.766450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:18.768373 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:18.768391 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:18.769536 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:18.769543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:18.769813 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:18.770564 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:18.770593 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:18.773233 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:18.776372 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:18.776384 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:18.776415 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:18.777170 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:18.777563 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:18.777570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:18.777576 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:18.777853 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:18.778395 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:18.778404 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:18.778893 (+ 489us) client_negotiation.cc:769] Sending connection context
0807 04:09:18.779239 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 04:09:18.779868 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":405,"threads_started":1}
I20260807 04:09:18.783145 20143 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38521
I20260807 04:09:18.783468 20143 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
I20260807 04:09:18.784464 20143 heartbeater.cc:511] Master 127.18.193.62:38521 requested a full tablet report, sending...
I20260807 04:09:18.786123 19808 ts_manager.cc:194] Registered new tserver with Master: 21d80a7207ff40538f101b009230eadd (127.18.193.2:45875)
I20260807 04:09:18.787397 19808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:36185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:18.808218 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38521
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:19.218575 20150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:19.219024 20150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:19.219278 20150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:19.229338 20150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:19.229506 20150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:19.229707 20150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:19.242265 20150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:19.243712 20150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:19.245534 20150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 04:09:19.248782 20005 heartbeater.cc:503] Master 127.18.193.62:38521 was elected leader, sending a full tablet report...
W20260807 04:09:19.263710 20156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:19.264091 20155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:19.264853 20158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:19.266047 20150 server_base.cc:1034] running on GCE node
I20260807 04:09:19.266805 20150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:19.268208 20150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:19.269485 20150 hybrid_clock.cc:648] HybridClock initialized: now 1786075759269364 us; error 112 us; skew 500 ppm
I20260807 04:09:19.269922 20150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:19.274242 20150 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:19.276242 20150 webserver.cc:460] Webserver started at http://127.18.193.3:44373/ using document root <none> and password file <none>
I20260807 04:09:19.277292 20150 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:19.277541 20150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:19.277961 20150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:19.281786 20150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "90d50309908a42769241e06753ac53a6"
format_stamp: "Formatted at 2026-08-07 04:09:19 on dist-test-slave-8n49"
server_key: "befb92d3fabf0ffc82f6ad1570c5d811"
server_key_iv: "a853f24184fb9cf4faaef7586c6539cf"
server_key_version: "encryptionkey@0"
I20260807 04:09:19.282902 20150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "90d50309908a42769241e06753ac53a6"
format_stamp: "Formatted at 2026-08-07 04:09:19 on dist-test-slave-8n49"
server_key: "befb92d3fabf0ffc82f6ad1570c5d811"
server_key_iv: "a853f24184fb9cf4faaef7586c6539cf"
server_key_version: "encryptionkey@0"
I20260807 04:09:19.290340 20150 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260807 04:09:19.295008 20165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:19.296660 20150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:09:19.296901 20150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "90d50309908a42769241e06753ac53a6"
format_stamp: "Formatted at 2026-08-07 04:09:19 on dist-test-slave-8n49"
server_key: "befb92d3fabf0ffc82f6ad1570c5d811"
server_key_iv: "a853f24184fb9cf4faaef7586c6539cf"
server_key_version: "encryptionkey@0"
I20260807 04:09:19.297183 20150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:19.338990 20150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:19.345196 20150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:19.345674 20150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:19.347236 20150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:19.349573 20150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:19.349730 20150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:19.349974 20150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:19.350109 20150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:19.423311 20150 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:43001
I20260807 04:09:19.423489 20282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:43001 every 8 connection(s)
I20260807 04:09:19.424971 20150 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:19.430748 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 20150
I20260807 04:09:19.431046 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:19.431568 19204 external_mini_cluster.cc:1428] Setting key 94d1b8f9d09525d6a8dc873f5aeff23b
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:19.451085 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.428347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:50559 (local address 127.18.193.62:38521)
0807 04:09:19.429012 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.429017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.430249 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 04:09:19.433159 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.433163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.433234 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.433356 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:19.435564 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.436495 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.438347 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.438919 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.439071 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.443309 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:19.443314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:19.443318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:19.443352 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:19.447380 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.448304 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.448312 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.448319 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.448498 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.449109 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.449113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.449115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.449449 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:19.449621 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.450546 (+ 925us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.450739 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":368,"threads_started":1}
I20260807 04:09:19.452250 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.429037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38521 (local address 127.18.193.3:50559)
0807 04:09:19.430067 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:19.430109 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:19.432678 (+ 2569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:19.433659 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:19.433671 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:19.434470 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:19.435379 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.435400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.436716 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.436720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.438184 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.438197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.439256 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.439274 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.439622 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.440417 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:19.440505 (+ 88us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:19.443072 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:19.447572 (+ 4500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:19.447579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:19.447593 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:19.448151 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:19.448670 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:19.448676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:19.448684 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:19.448932 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:19.449666 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:19.449673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:19.450058 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 04:09:19.450367 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 04:09:19.450834 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":556,"threads_started":1}
I20260807 04:09:19.453954 20283 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38521
I20260807 04:09:19.454391 20283 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:19.455837 20283 heartbeater.cc:511] Master 127.18.193.62:38521 requested a full tablet report, sending...
I20260807 04:09:19.457445 19808 ts_manager.cc:194] Registered new tserver with Master: 90d50309908a42769241e06753ac53a6 (127.18.193.3:43001)
I20260807 04:09:19.458338 19808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:50559
I20260807 04:09:19.461112 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:09:19.488219 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.469357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.18.193.62:38521)
0807 04:09:19.469568 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.469573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.470458 (+ 885us) server_negotiation.cc:407] Connection header received
0807 04:09:19.470708 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.470713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.470782 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.470893 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:19.473649 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.474708 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.476621 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.477291 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.477462 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.481216 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:19.481224 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:19.481229 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:19.481279 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:19.485135 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.485912 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.485919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.485922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.486111 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.486535 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.486541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.486551 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.486940 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:19.487242 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.487583 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.487815 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 04:09:19.495720 19808 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:19.505563 19808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:19.547567 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.523115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.18.193.1:39665)
0807 04:09:19.535587 (+ 12472us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.535602 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.535643 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:09:19.535774 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.535786 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.536202 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.537835 (+ 1633us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:19.538363 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.540411 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.543849 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.545817 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.546021 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.546307 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.546900 (+ 593us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.547141 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12169,"thread_start_us":12467,"threads_started":1}
I20260807 04:09:19.547683 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.521976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39665 (local address 127.0.0.1:50762)
0807 04:09:19.534284 (+ 12308us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:19.534340 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:19.534539 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:19.536493 (+ 1954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:19.536504 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:19.536539 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:19.537547 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.537567 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.540592 (+ 3025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.540599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.543673 (+ 3074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.543686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.546376 (+ 2690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.546383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.546614 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.546671 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 04:09:19.546812 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 04:09:19.547147 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12029,"thread_start_us":12915,"threads_started":1}
I20260807 04:09:19.548509 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.522255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:43001 (local address 127.0.0.1:43070)
0807 04:09:19.535253 (+ 12998us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:19.535280 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:19.535394 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:19.537748 (+ 2354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:19.537758 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:19.537799 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:19.538985 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.539004 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.541294 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.541300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.545082 (+ 3782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.545098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.547481 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.547488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.547773 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.547823 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 04:09:19.547894 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 04:09:19.548027 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12842,"mutex_wait_us":6,"thread_start_us":11551,"threads_started":1}
I20260807 04:09:19.549181 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.523152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.18.193.3:43001)
0807 04:09:19.536875 (+ 13723us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.536884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.536936 (+ 52us) server_negotiation.cc:407] Connection header received
0807 04:09:19.537089 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.537097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.537343 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.538185 (+ 842us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:19.539298 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.541120 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.545321 (+ 4201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.547132 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.547978 (+ 846us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.548266 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.548351 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.548530 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13461,"thread_start_us":13113,"threads_started":1}
I20260807 04:09:19.553646 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.535743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.18.193.2:45875)
0807 04:09:19.537393 (+ 1650us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.537403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.539435 (+ 2032us) server_negotiation.cc:407] Connection header received
0807 04:09:19.539818 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.539826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.540159 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.540365 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:19.542225 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.544366 (+ 2141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.548172 (+ 3806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.551624 (+ 3452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.551798 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.552019 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.552723 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.552954 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1458,"thread_start_us":1006,"threads_started":1}
I20260807 04:09:19.553500 19937 tablet_service.cc:1467] Processing CreateTablet for tablet 4f8803818d3442c6a066b5a37844482b (DEFAULT_TABLE table=test-table [id=29e1ed05455243aa8d162adc80625190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:19.553736 20216 tablet_service.cc:1467] Processing CreateTablet for tablet 4f8803818d3442c6a066b5a37844482b (DEFAULT_TABLE table=test-table [id=29e1ed05455243aa8d162adc80625190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:19.554548 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.537042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45875 (local address 127.0.0.1:38154)
0807 04:09:19.539278 (+ 2236us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:19.539324 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:19.539603 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:19.540729 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:19.540736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:19.540763 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:19.541964 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.541984 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.544536 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.544542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.547936 (+ 3394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.547952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.551997 (+ 4045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.552007 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.552278 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.552314 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:09:19.552446 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 04:09:19.554009 (+ 1563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1228,"thread_start_us":788,"threads_started":1}
I20260807 04:09:19.555388 20216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f8803818d3442c6a066b5a37844482b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:19.555493 19937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f8803818d3442c6a066b5a37844482b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:19.559552 20075 tablet_service.cc:1467] Processing CreateTablet for tablet 4f8803818d3442c6a066b5a37844482b (DEFAULT_TABLE table=test-table [id=29e1ed05455243aa8d162adc80625190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:19.561543 20075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f8803818d3442c6a066b5a37844482b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:19.570526 20304 tablet_bootstrap.cc:492] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3: Bootstrap starting.
I20260807 04:09:19.576328 20305 tablet_bootstrap.cc:492] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6: Bootstrap starting.
I20260807 04:09:19.577744 20304 tablet_bootstrap.cc:654] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:19.580397 20304 log.cc:824] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:19.581881 20306 tablet_bootstrap.cc:492] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd: Bootstrap starting.
I20260807 04:09:19.582381 20305 tablet_bootstrap.cc:654] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:19.584651 20305 log.cc:824] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:19.586599 20306 tablet_bootstrap.cc:654] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:19.587946 20304 tablet_bootstrap.cc:492] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3: No bootstrap required, opened a new log
I20260807 04:09:19.588162 20306 log.cc:824] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:19.588302 20304 ts_tablet_manager.cc:1397] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260807 04:09:19.591363 20305 tablet_bootstrap.cc:492] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6: No bootstrap required, opened a new log
I20260807 04:09:19.591675 20305 ts_tablet_manager.cc:1397] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.007s
I20260807 04:09:19.591900 20306 tablet_bootstrap.cc:492] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd: No bootstrap required, opened a new log
I20260807 04:09:19.592286 20306 ts_tablet_manager.cc:1397] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 04:09:19.599136 20304 raft_consensus.cc:348] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.599684 20304 raft_consensus.cc:374] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:19.599810 20304 raft_consensus.cc:729] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78815cef9e6f4d8cb410787bf93430a3, State: Initialized, Role: FOLLOWER
I20260807 04:09:19.600512 20304 consensus_queue.cc:260] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.607653 20306 raft_consensus.cc:348] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.607653 20305 raft_consensus.cc:348] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.608374 20306 raft_consensus.cc:374] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:19.608374 20305 raft_consensus.cc:374] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:19.608546 20306 raft_consensus.cc:729] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d80a7207ff40538f101b009230eadd, State: Initialized, Role: FOLLOWER
I20260807 04:09:19.608546 20305 raft_consensus.cc:729] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d50309908a42769241e06753ac53a6, State: Initialized, Role: FOLLOWER
I20260807 04:09:19.609434 20305 consensus_queue.cc:260] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.609434 20306 consensus_queue.cc:260] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.618911 20283 heartbeater.cc:503] Master 127.18.193.62:38521 was elected leader, sending a full tablet report...
I20260807 04:09:19.619344 20305 ts_tablet_manager.cc:1428] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6: Time spent starting tablet: real 0.027s user 0.019s sys 0.008s
I20260807 04:09:19.624228 20306 ts_tablet_manager.cc:1428] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd: Time spent starting tablet: real 0.032s user 0.018s sys 0.009s
I20260807 04:09:19.624454 20304 ts_tablet_manager.cc:1428] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3: Time spent starting tablet: real 0.036s user 0.012s sys 0.019s
I20260807 04:09:19.626482 20312 raft_consensus.cc:482] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:19.626638 20143 heartbeater.cc:503] Master 127.18.193.62:38521 was elected leader, sending a full tablet report...
I20260807 04:09:19.627184 20312 raft_consensus.cc:504] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.630180 20312 leader_election.cc:288] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78815cef9e6f4d8cb410787bf93430a3 (127.18.193.1:39665), 90d50309908a42769241e06753ac53a6 (127.18.193.3:43001)
I20260807 04:09:19.642928 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.630555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39665 (local address 127.18.193.2:41563)
0807 04:09:19.632736 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:19.632763 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:19.633024 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:19.633912 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:19.633917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:19.633947 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:19.634668 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.634683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.636739 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.636747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.640126 (+ 3379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.640141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.642121 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.642131 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.642270 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.642298 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 04:09:19.642381 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 04:09:19.642481 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1859,"thread_start_us":2689,"threads_started":1}
I20260807 04:09:19.642911 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.631016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:41563 (local address 127.18.193.1:39665)
0807 04:09:19.631308 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.631315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.632847 (+ 1532us) server_negotiation.cc:407] Connection header received
0807 04:09:19.633320 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.633326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.633417 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.633528 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:19.634857 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.636604 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.640324 (+ 3720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.641799 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.642040 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.642145 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.642438 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.642581 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 04:09:19.644647 19957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8803818d3442c6a066b5a37844482b" candidate_uuid: "21d80a7207ff40538f101b009230eadd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78815cef9e6f4d8cb410787bf93430a3" is_pre_election: true
I20260807 04:09:19.645749 19957 raft_consensus.cc:2454] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21d80a7207ff40538f101b009230eadd in term 0.
I20260807 04:09:19.646838 20028 leader_election.cc:302] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21d80a7207ff40538f101b009230eadd, 78815cef9e6f4d8cb410787bf93430a3; no voters:
I20260807 04:09:19.647280 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.634682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:51249 (local address 127.18.193.3:43001)
0807 04:09:19.634947 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.634957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.637594 (+ 2637us) server_negotiation.cc:407] Connection header received
0807 04:09:19.637846 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.637870 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.638092 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.638217 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:19.639550 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.640717 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.643832 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.645733 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.646161 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.646267 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.646798 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.646928 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:09:19.647537 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.635546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:43001 (local address 127.18.193.2:51249)
0807 04:09:19.637458 (+ 1912us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:19.637481 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:19.637634 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:19.638246 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:19.638256 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:19.638303 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:19.639354 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.639367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.640891 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.640896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.643630 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:19.643646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.646247 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.646322 (+ 75us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:19.646632 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.646664 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:09:19.646772 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 04:09:19.646934 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1652,"thread_start_us":727,"threads_started":1}
I20260807 04:09:19.647708 20314 raft_consensus.cc:2781] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:19.647962 20314 raft_consensus.cc:482] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:19.648310 20314 raft_consensus.cc:3037] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:19.648777 20237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8803818d3442c6a066b5a37844482b" candidate_uuid: "21d80a7207ff40538f101b009230eadd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d50309908a42769241e06753ac53a6" is_pre_election: true
I20260807 04:09:19.649307 20237 raft_consensus.cc:2454] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21d80a7207ff40538f101b009230eadd in term 0.
I20260807 04:09:19.652812 20314 raft_consensus.cc:504] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.654145 20314 leader_election.cc:288] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [CANDIDATE]: Term 1 election: Requested vote from peers 78815cef9e6f4d8cb410787bf93430a3 (127.18.193.1:39665), 90d50309908a42769241e06753ac53a6 (127.18.193.3:43001)
I20260807 04:09:19.654676 20237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8803818d3442c6a066b5a37844482b" candidate_uuid: "21d80a7207ff40538f101b009230eadd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d50309908a42769241e06753ac53a6"
I20260807 04:09:19.654654 19957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8803818d3442c6a066b5a37844482b" candidate_uuid: "21d80a7207ff40538f101b009230eadd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78815cef9e6f4d8cb410787bf93430a3"
I20260807 04:09:19.654896 19957 raft_consensus.cc:3037] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:19.654968 20237 raft_consensus.cc:3037] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:19.659895 20237 raft_consensus.cc:2454] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21d80a7207ff40538f101b009230eadd in term 1.
I20260807 04:09:19.659883 19957 raft_consensus.cc:2454] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21d80a7207ff40538f101b009230eadd in term 1.
I20260807 04:09:19.660395 20028 leader_election.cc:302] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21d80a7207ff40538f101b009230eadd, 78815cef9e6f4d8cb410787bf93430a3; no voters:
I20260807 04:09:19.660820 20314 raft_consensus.cc:2781] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:19.661159 20314 raft_consensus.cc:686] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [term 1 LEADER]: Becoming Leader. State: Replica: 21d80a7207ff40538f101b009230eadd, State: Running, Role: LEADER
I20260807 04:09:19.661859 20314 consensus_queue.cc:237] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } }
I20260807 04:09:19.667830 19807 catalog_manager.cc:5495] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd reported cstate change: term changed from 0 to 1, leader changed from <none> to 21d80a7207ff40538f101b009230eadd (127.18.193.2). New cstate: current_term: 1 leader_uuid: "21d80a7207ff40538f101b009230eadd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78815cef9e6f4d8cb410787bf93430a3" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90d50309908a42769241e06753ac53a6" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 43001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21d80a7207ff40538f101b009230eadd" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45875 } health_report { overall_health: HEALTHY } } }
W20260807 04:09:19.677104 20284 tablet.cc:2369] T 4f8803818d3442c6a066b5a37844482b P 90d50309908a42769241e06753ac53a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:19.704308 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.683004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.18.193.1:39665)
0807 04:09:19.683274 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.683280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.684468 (+ 1188us) server_negotiation.cc:407] Connection header received
0807 04:09:19.684737 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.684742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.684809 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.684943 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:19.687635 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.688529 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.690384 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.691092 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.691220 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.697123 (+ 5903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:19.697140 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:19.697159 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:19.697198 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:19.700970 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.701818 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.701827 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.701829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.702011 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.702437 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.702443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.702446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.703000 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:19.703371 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.703575 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.703748 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260807 04:09:19.716413 20006 tablet.cc:2369] T 4f8803818d3442c6a066b5a37844482b P 78815cef9e6f4d8cb410787bf93430a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
W20260807 04:09:19.759475 20144 tablet.cc:2369] T 4f8803818d3442c6a066b5a37844482b P 21d80a7207ff40538f101b009230eadd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:09:19.766108 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.744830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.18.193.1:39665)
0807 04:09:19.745265 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.745276 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.745865 (+ 589us) server_negotiation.cc:407] Connection header received
0807 04:09:19.745986 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.745990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.746063 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.746157 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:19.748261 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.749183 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.750592 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.751029 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.751175 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.757124 (+ 5949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:19.757133 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:19.757139 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:19.757191 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:19.762320 (+ 5129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.763106 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.763113 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.763118 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.763487 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.764033 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.764039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.764043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.764521 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:19.765055 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.765231 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.765540 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:19.823514 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.805322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.18.193.1:39665)
0807 04:09:19.805610 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.805615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.805738 (+ 123us) server_negotiation.cc:407] Connection header received
0807 04:09:19.805898 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.805903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.805971 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.806063 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:19.808297 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.809158 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.810835 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.811349 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.811461 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.817056 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:19.817062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:19.817065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:19.817115 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:19.820358 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.821246 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.821251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.821253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.821437 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.821868 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.821872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.821874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.822350 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:19.822639 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.822862 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.823100 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260807 04:09:19.825717 19916 tablet_service.cc:2908] Rejecting scan request for tablet 4f8803818d3442c6a066b5a37844482b: Uninitialized: clean time has not yet been initialized
I20260807 04:09:19.829707 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 19872
I20260807 04:09:19.845351 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.830533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.18.193.1:39665)
0807 04:09:19.830722 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.830727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.831004 (+ 277us) server_negotiation.cc:407] Connection header received
0807 04:09:19.831118 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.831122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.831197 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.831285 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:19.832812 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.833558 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.835166 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.835857 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.836057 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.839106 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:19.839112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:19.839115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:19.839146 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:19.842272 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.842939 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.842943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.842945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.843063 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.843600 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.843604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.843606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.844046 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:19.844285 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.844747 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.845007 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:09:19.846227 19999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:19.944284 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 19872
I20260807 04:09:19.964617 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20011
Aug 07 04:09:19 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:09:19.980405 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:19.965437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.18.193.2:45875)
0807 04:09:19.965654 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:19.965661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:19.965683 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:09:19.965898 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:19.965903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:19.965986 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:19.966101 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:19.967346 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.968074 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.969560 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:19.970005 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:19.970154 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:19.974313 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:19.974329 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:19.974345 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:19.974389 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:19.977551 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.978226 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.978233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.978237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.978368 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:19.978776 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:19.978785 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:19.978787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:19.979154 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:19.979465 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:19.979754 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 04:09:19.980009 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:09:19.981184 20138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:20.075738 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 20011
I20260807 04:09:20.095703 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20150
Aug 07 04:09:20 dist-test-slave-8n49 krb5kdc[19760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:09:20.112166 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:20.096603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.18.193.3:43001)
0807 04:09:20.096791 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:20.096795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:20.096814 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:09:20.096932 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:20.096936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:20.097009 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:20.097104 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:20.098528 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:20.099281 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:20.101118 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:20.101657 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:20.101771 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:20.106151 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:20.106166 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:20.106183 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:20.106218 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:20.109489 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:20.110187 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:20.110194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:20.110196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:20.110332 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:20.110694 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:20.110697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:20.110699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:20.111043 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:20.111272 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:20.111549 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 04:09:20.111732 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:09:20.113003 20278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:20.206127 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 20150
I20260807 04:09:20.224764 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 19773
I20260807 04:09:20.226054 19839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:20.339911 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 19773
2026-08-07T04:09:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5411 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:20 dist-test-slave-8n49 krb5kdc[20335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:20 dist-test-slave-8n49 krb5kdc[20335](info): set up 2 sockets
Aug 07 04:09:20 dist-test-slave-8n49 krb5kdc[20335](info): commencing operation
krb5kdc: starting...
W20260807 04:09:21.570549 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:21 dist-test-slave-8n49 krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:21.793291 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41079
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:41079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:22.135921 20349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:22.136353 20349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:22.136543 20349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:22.144203 20349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:22.144372 20349 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 04:09:22.144495 20349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:22.144637 20349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:22.144737 20349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:22.144830 20349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:22.156199 20349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:41079
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:22.157718 20349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:22.159168 20349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:22.171614 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:22.171725 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:22.171746 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:22.172662 20349 server_base.cc:1034] running on GCE node
I20260807 04:09:22.173516 20349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:22.175143 20349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:22.176420 20349 hybrid_clock.cc:648] HybridClock initialized: now 1786075762176361 us; error 35 us; skew 500 ppm
I20260807 04:09:22.176843 20349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:22 dist-test-slave-8n49 krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075762, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:22.179988 20349 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:22.181592 20349 webserver.cc:460] Webserver started at http://127.18.193.62:41931/ using document root <none> and password file <none>
I20260807 04:09:22.182382 20349 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:22.182472 20349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:22.182749 20349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:22.185918 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "f59bb9c46c3a4897abc631bb8ace809e"
format_stamp: "Formatted at 2026-08-07 04:09:22 on dist-test-slave-8n49"
server_key: "6c70838d03158dcf55ff3a2ec8f7861b"
server_key_iv: "002f527f817d377ff7a8a46e7a4f85fb"
server_key_version: "encryptionkey@0"
I20260807 04:09:22.186668 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "f59bb9c46c3a4897abc631bb8ace809e"
format_stamp: "Formatted at 2026-08-07 04:09:22 on dist-test-slave-8n49"
server_key: "6c70838d03158dcf55ff3a2ec8f7861b"
server_key_iv: "002f527f817d377ff7a8a46e7a4f85fb"
server_key_version: "encryptionkey@0"
I20260807 04:09:22.191954 20349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:09:22.195643 20364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:22.197079 20349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:22.197228 20349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "f59bb9c46c3a4897abc631bb8ace809e"
format_stamp: "Formatted at 2026-08-07 04:09:22 on dist-test-slave-8n49"
server_key: "6c70838d03158dcf55ff3a2ec8f7861b"
server_key_iv: "002f527f817d377ff7a8a46e7a4f85fb"
server_key_version: "encryptionkey@0"
I20260807 04:09:22.197409 20349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:22.213420 20349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:22.217883 20349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:22.218127 20349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:22.243330 20349 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:41079
I20260807 04:09:22.243367 20428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:41079 every 8 connection(s)
I20260807 04:09:22.244866 20349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:22.251081 20429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:22.254881 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 20349
I20260807 04:09:22.255173 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:22.256035 19204 external_mini_cluster.cc:1428] Setting key 465aa9a7293fa7e57fd51004e2ddac31
I20260807 04:09:22.264899 20429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e: Bootstrap starting.
I20260807 04:09:22.268244 20429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:22.269500 20429 log.cc:824] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:22.273137 20429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e: No bootstrap required, opened a new log
Aug 07 04:09:22 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:22.281309 20429 raft_consensus.cc:348] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f59bb9c46c3a4897abc631bb8ace809e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41079 } }
I20260807 04:09:22.281730 20429 raft_consensus.cc:374] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:22.281853 20429 raft_consensus.cc:729] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f59bb9c46c3a4897abc631bb8ace809e, State: Initialized, Role: FOLLOWER
I20260807 04:09:22.282428 20429 consensus_queue.cc:260] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f59bb9c46c3a4897abc631bb8ace809e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41079 } }
I20260807 04:09:22.282717 20429 raft_consensus.cc:388] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:22.282877 20429 raft_consensus.cc:482] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:22.283067 20429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:22.283418 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:22.258438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.18.193.62:41079)
0807 04:09:22.259372 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:22.259392 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:22.259447 (+ 55us) server_negotiation.cc:407] Connection header received
0807 04:09:22.261349 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:22.261374 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:22.261808 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:22.262266 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:22.263789 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.266052 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.267667 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.268704 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.268961 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:22.274602 (+ 5641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:22.274616 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:22.274632 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:22.274686 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:22.278340 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:22.279155 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:22.279171 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:22.279179 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:22.279366 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:22.279818 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:22.279823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:22.279827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:22.280482 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:22.280891 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:22.281428 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 04:09:22.281989 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":388,"threads_started":1}
I20260807 04:09:22.286005 20429 raft_consensus.cc:504] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f59bb9c46c3a4897abc631bb8ace809e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41079 } }
I20260807 04:09:22.286546 20429 leader_election.cc:302] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f59bb9c46c3a4897abc631bb8ace809e; no voters:
I20260807 04:09:22.287250 20429 leader_election.cc:288] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:22.287412 20434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:22.288434 20434 raft_consensus.cc:686] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [term 1 LEADER]: Becoming Leader. State: Replica: f59bb9c46c3a4897abc631bb8ace809e, State: Running, Role: LEADER
I20260807 04:09:22.289137 20434 consensus_queue.cc:237] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f59bb9c46c3a4897abc631bb8ace809e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41079 } }
I20260807 04:09:22.289737 20429 sys_catalog.cc:564] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:22.292306 20435 sys_catalog.cc:455] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f59bb9c46c3a4897abc631bb8ace809e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f59bb9c46c3a4897abc631bb8ace809e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41079 } } }
I20260807 04:09:22.292870 20435 sys_catalog.cc:458] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:22.293478 20436 sys_catalog.cc:455] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f59bb9c46c3a4897abc631bb8ace809e. Latest consensus state: current_term: 1 leader_uuid: "f59bb9c46c3a4897abc631bb8ace809e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f59bb9c46c3a4897abc631bb8ace809e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41079 } } }
I20260807 04:09:22.293846 20436 sys_catalog.cc:458] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:22.299516 20442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:22.304872 20442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:22.314235 20442 catalog_manager.cc:1347] Generated new cluster ID: f0fd77766888452dae98d11eeb549b27
I20260807 04:09:22.314378 20442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:22.346294 20442 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:22.348527 20442 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:22.365911 20442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e: Generated new TSK 0
I20260807 04:09:22.367208 20442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:22.433808 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41079
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:22.804441 20457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:22.804814 20457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:22.804916 20457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:22.812420 20457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:22.812595 20457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:22.812713 20457 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:09:22.812803 20457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:22.823307 20457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:41079
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:22.824836 20457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:22.826184 20457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:22.841267 20466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:22.841429 20462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:22.841367 20464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:22.842187 20457 server_base.cc:1034] running on GCE node
I20260807 04:09:22.842897 20457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:22.843840 20457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:22.845134 20457 hybrid_clock.cc:648] HybridClock initialized: now 1786075762845047 us; error 56 us; skew 500 ppm
I20260807 04:09:22.845501 20457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:22 dist-test-slave-8n49 krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075762, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:22.852231 20457 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:22.854197 20457 webserver.cc:460] Webserver started at http://127.18.193.1:35883/ using document root <none> and password file <none>
I20260807 04:09:22.855068 20457 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:22.855235 20457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:22.855643 20457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:22.859181 20457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b"
format_stamp: "Formatted at 2026-08-07 04:09:22 on dist-test-slave-8n49"
server_key: "b7d1a50116b28084089cf5cf0835894e"
server_key_iv: "bfb5b3fb8351c6578802782fa1eb908b"
server_key_version: "encryptionkey@0"
I20260807 04:09:22.860129 20457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b"
format_stamp: "Formatted at 2026-08-07 04:09:22 on dist-test-slave-8n49"
server_key: "b7d1a50116b28084089cf5cf0835894e"
server_key_iv: "bfb5b3fb8351c6578802782fa1eb908b"
server_key_version: "encryptionkey@0"
I20260807 04:09:22.866442 20457 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 04:09:22.870558 20473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:22.872141 20457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:09:22.872346 20457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b"
format_stamp: "Formatted at 2026-08-07 04:09:22 on dist-test-slave-8n49"
server_key: "b7d1a50116b28084089cf5cf0835894e"
server_key_iv: "bfb5b3fb8351c6578802782fa1eb908b"
server_key_version: "encryptionkey@0"
I20260807 04:09:22.872644 20457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:22.893082 20457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:22.898051 20457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:22.898370 20457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:22.901688 20457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:22.901777 20457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:22.902007 20457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:22.902088 20457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:09:22 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075762, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:22.938043 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:22.902416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:39917 (local address 127.18.193.62:41079)
0807 04:09:22.913562 (+ 11146us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:22.913567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:22.913586 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:09:22.913643 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:22.913647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:22.913754 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:22.913896 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:22.917008 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.918045 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.921425 (+ 3380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.922204 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.922335 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:22.929048 (+ 6713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:22.929059 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:22.929067 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:22.929126 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:22.933241 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:22.934492 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:22.934501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:22.934510 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:22.934691 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:22.935383 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:22.935389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:22.935392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:22.935699 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:22.935827 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:22.937158 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0807 04:09:22.937485 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11016,"thread_start_us":10897,"threads_started":1}
I20260807 04:09:22.939661 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:22.903373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41079 (local address 127.18.193.1:39917)
0807 04:09:22.907068 (+ 3695us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:22.907142 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:22.910585 (+ 3443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:22.914294 (+ 3709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:22.914313 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:22.915428 (+ 1115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:22.916736 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:22.916774 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.918269 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.918278 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:22.921190 (+ 2912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:22.921210 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.922840 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.922850 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:22.923245 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:22.924212 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:22.924263 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:22.928685 (+ 4422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:22.933451 (+ 4766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:22.933467 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:22.933496 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:22.934311 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:22.934873 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:22.934879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:22.934891 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:22.935221 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:22.935847 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:22.935860 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:22.936509 (+ 649us) client_negotiation.cc:769] Sending connection context
0807 04:09:22.937003 (+ 494us) client_negotiation.cc:240] Negotiation successful
0807 04:09:22.937807 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3291,"thread_start_us":2996,"threads_started":1}
I20260807 04:09:22.949748 20457 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:34901
I20260807 04:09:22.949872 20596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:34901 every 8 connection(s)
I20260807 04:09:22.951102 20457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:22.953390 20597 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41079
I20260807 04:09:22.953665 20597 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:22.954393 20597 heartbeater.cc:511] Master 127.18.193.62:41079 requested a full tablet report, sending...
I20260807 04:09:22.956990 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 20457
I20260807 04:09:22.957162 20381 ts_manager.cc:194] Registered new tserver with Master: 0ddb7dcf9b5d4f0082a262b020c1b75b (127.18.193.1:34901)
I20260807 04:09:22.957232 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:22.958052 19204 external_mini_cluster.cc:1428] Setting key 9dfb8f2b3c98aaae22b6dfe5221fa364
I20260807 04:09:22.959414 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:39917
I20260807 04:09:22.963889 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:22.947912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:39501 (local address 127.18.193.62:41079)
0807 04:09:22.948129 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:22.948137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:22.948875 (+ 738us) server_negotiation.cc:407] Connection header received
0807 04:09:22.949129 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:22.949133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:22.949198 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:22.949304 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:22.950549 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.951313 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.953857 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.954400 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.954569 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:22.956765 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:22.956772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:22.956777 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:22.956819 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:22.960695 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:22.961740 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:22.961746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:22.961749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:22.961905 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:22.962399 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:22.962404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:22.962407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:22.962766 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:22.962926 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:22.963470 (+ 544us) server_negotiation.cc:299] Negotiation successful
0807 04:09:22.963653 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:09:22.964500 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:22.947962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41079 (local address 127.18.193.1:39501)
0807 04:09:22.948751 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:22.948774 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:22.948967 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:22.949338 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:22.949342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:22.950083 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:22.950393 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:22.950403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.951441 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.951448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:22.953709 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:22.953726 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:22.954608 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:22.954615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:22.954801 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:22.955573 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:22.955597 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:22.956534 (+ 937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:22.960866 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:22.960873 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:22.960877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:22.961572 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:22.962040 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:22.962054 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:22.962062 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:22.962250 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:22.962919 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:22.962928 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:22.963175 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 04:09:22.963350 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 04:09:22.963752 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":436,"threads_started":1}
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:23.008882 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41079
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:23.343621 20602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:23.343926 20602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:23.344110 20602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:23.351648 20602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:23.351784 20602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:23.351931 20602 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:09:23.352092 20602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:23.363061 20602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:41079
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:23.364528 20602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:23.365974 20602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:23.383811 20607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:23.384761 20610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:23.385165 20608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:23.385565 20602 server_base.cc:1034] running on GCE node
I20260807 04:09:23.386338 20602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:23.387374 20602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:23.388671 20602 hybrid_clock.cc:648] HybridClock initialized: now 1786075763388586 us; error 64 us; skew 500 ppm
I20260807 04:09:23.388996 20602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:23 dist-test-slave-8n49 krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075763, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:23.392360 20602 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:23.393898 20602 webserver.cc:460] Webserver started at http://127.18.193.2:42515/ using document root <none> and password file <none>
I20260807 04:09:23.394646 20602 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:23.394784 20602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:23.395128 20602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:23.398454 20602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "adc2e8bd4cd744e7ab647afb84990b27"
format_stamp: "Formatted at 2026-08-07 04:09:23 on dist-test-slave-8n49"
server_key: "aa2fd8e0bd5e7ef913a715c2cd2907fb"
server_key_iv: "5ca69c3fdccf538ef787e30563057c99"
server_key_version: "encryptionkey@0"
I20260807 04:09:23.399320 20602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "adc2e8bd4cd744e7ab647afb84990b27"
format_stamp: "Formatted at 2026-08-07 04:09:23 on dist-test-slave-8n49"
server_key: "aa2fd8e0bd5e7ef913a715c2cd2907fb"
server_key_iv: "5ca69c3fdccf538ef787e30563057c99"
server_key_version: "encryptionkey@0"
I20260807 04:09:23.405093 20602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 04:09:23.408861 20618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:23.410248 20602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:09:23.410437 20602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "adc2e8bd4cd744e7ab647afb84990b27"
format_stamp: "Formatted at 2026-08-07 04:09:23 on dist-test-slave-8n49"
server_key: "aa2fd8e0bd5e7ef913a715c2cd2907fb"
server_key_iv: "5ca69c3fdccf538ef787e30563057c99"
server_key_version: "encryptionkey@0"
I20260807 04:09:23.410683 20602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:23.427870 20602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:23.432998 20602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:23.433307 20602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:23.436115 20602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:23.436233 20602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:23.436443 20602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:23.436518 20602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:09:23 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075763, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:23.464533 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:23.437824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41079 (local address 127.18.193.2:38769)
0807 04:09:23.438869 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:23.438923 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:23.441320 (+ 2397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:23.442200 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:23.442220 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:23.443091 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:23.444085 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:23.444111 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.445519 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.445528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:23.448585 (+ 3057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:23.448609 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.449813 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.449821 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:23.450261 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:23.451227 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:23.451267 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:23.454633 (+ 3366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:23.458336 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:23.458348 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:23.458367 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:23.459213 (+ 846us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:23.459687 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:23.459693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:23.459696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:23.460068 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:23.460935 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:23.460941 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:23.461538 (+ 597us) client_negotiation.cc:769] Sending connection context
0807 04:09:23.462026 (+ 488us) client_negotiation.cc:240] Negotiation successful
0807 04:09:23.462706 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":377,"threads_started":1}
I20260807 04:09:23.465360 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:23.437133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:38769 (local address 127.18.193.62:41079)
0807 04:09:23.437300 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:23.437305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:23.439053 (+ 1748us) server_negotiation.cc:407] Connection header received
0807 04:09:23.441678 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:23.441683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:23.441784 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:23.441898 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:23.444293 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.445324 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.448811 (+ 3487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.449464 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.449637 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:23.454882 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:23.454887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:23.454890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:23.454923 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:23.458168 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:23.459396 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:23.459404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:23.459410 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:23.459577 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:23.460253 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:23.460261 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:23.460266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:23.460699 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:23.460858 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:23.464796 (+ 3938us) server_negotiation.cc:299] Negotiation successful
0807 04:09:23.465069 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 04:09:23.488281 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:23.471725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:51051 (local address 127.18.193.62:41079)
0807 04:09:23.471986 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:23.471994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:23.472351 (+ 357us) server_negotiation.cc:407] Connection header received
0807 04:09:23.472588 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:23.472596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:23.472699 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:23.472813 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:23.475073 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.476420 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.478854 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.479392 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.479534 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:23.482280 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:23.482286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:23.482289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:23.482326 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:23.485199 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:23.486191 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:23.486198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:23.486201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:23.486361 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:23.486738 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:23.486744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:23.486747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:23.487113 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:23.487273 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:23.487691 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 04:09:23.487921 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 04:09:23.488416 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:23.471596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41079 (local address 127.18.193.2:51051)
0807 04:09:23.472243 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:23.472261 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:23.472400 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:23.473052 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:23.473056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:23.474209 (+ 1153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:23.474799 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:23.474820 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.476722 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.476728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:23.478650 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:23.478672 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:23.479636 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:23.479645 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:23.479895 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:23.480782 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:23.480801 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:23.482071 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:23.485380 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:23.485383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:23.485387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:23.486045 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:23.486489 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:23.486493 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:23.486497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:23.486626 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:23.487247 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:23.487252 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:23.487440 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 04:09:23.487596 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 04:09:23.487948 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":338,"threads_started":1}
I20260807 04:09:23.494366 20602 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:44309
I20260807 04:09:23.494457 20741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:44309 every 8 connection(s)
I20260807 04:09:23.495595 20602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:23.497922 20742 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41079
I20260807 04:09:23.498216 20742 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:23.498898 20742 heartbeater.cc:511] Master 127.18.193.62:41079 requested a full tablet report, sending...
I20260807 04:09:23.500131 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 20602
I20260807 04:09:23.500339 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:23.500334 20381 ts_manager.cc:194] Registered new tserver with Master: adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2:44309)
I20260807 04:09:23.500898 19204 external_mini_cluster.cc:1428] Setting key 8005f2ca977454d3398d3fe8e7032dd1
I20260807 04:09:23.501444 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:38769
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:23.549918 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41079
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:23.939459 20746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:23.939780 20746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:23.939955 20746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:23.947881 20746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:23.948004 20746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:23.948177 20746 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:09:23.948297 20746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:23.958724 20746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:41079
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:23.960013 20746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:23.961442 20746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 04:09:23.963521 20597 heartbeater.cc:503] Master 127.18.193.62:41079 was elected leader, sending a full tablet report...
W20260807 04:09:23.975673 20756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:23.975714 20754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:23.975899 20753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:23.976946 20746 server_base.cc:1034] running on GCE node
I20260807 04:09:23.977639 20746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:23.978617 20746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:23.979840 20746 hybrid_clock.cc:648] HybridClock initialized: now 1786075763979771 us; error 42 us; skew 500 ppm
I20260807 04:09:23.980168 20746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:23 dist-test-slave-8n49 krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075763, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:23.983435 20746 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:23.985028 20746 webserver.cc:460] Webserver started at http://127.18.193.3:46805/ using document root <none> and password file <none>
I20260807 04:09:23.985754 20746 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:23.985870 20746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:23.986258 20746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:23.989353 20746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "92f89593f01a4c25bad0de7b5e7ad6cb"
format_stamp: "Formatted at 2026-08-07 04:09:23 on dist-test-slave-8n49"
server_key: "4c74422c0a91abdb9b494920b78fb206"
server_key_iv: "d946f63de48c72a24ba56afc6a3f3630"
server_key_version: "encryptionkey@0"
I20260807 04:09:23.990275 20746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "92f89593f01a4c25bad0de7b5e7ad6cb"
format_stamp: "Formatted at 2026-08-07 04:09:23 on dist-test-slave-8n49"
server_key: "4c74422c0a91abdb9b494920b78fb206"
server_key_iv: "d946f63de48c72a24ba56afc6a3f3630"
server_key_version: "encryptionkey@0"
I20260807 04:09:23.995697 20746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:09:23.999456 20763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:24.000921 20746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:24.001114 20746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "92f89593f01a4c25bad0de7b5e7ad6cb"
format_stamp: "Formatted at 2026-08-07 04:09:23 on dist-test-slave-8n49"
server_key: "4c74422c0a91abdb9b494920b78fb206"
server_key_iv: "d946f63de48c72a24ba56afc6a3f3630"
server_key_version: "encryptionkey@0"
I20260807 04:09:24.001370 20746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:24.012988 20746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:24.017786 20746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:24.018198 20746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:24.021216 20746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:24.021340 20746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:24.021526 20746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:24.021613 20746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:09:24 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075763, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:24.047627 20772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.021827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:38773 (local address 127.18.193.62:41079)
0807 04:09:24.024104 (+ 2277us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.024111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.024136 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:09:24.025833 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.025839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.025940 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.026066 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:24.028655 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.030044 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.032750 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.033511 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.033675 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.038466 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:24.038477 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:24.038484 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:24.038542 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:24.043108 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:24.044051 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:24.044065 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:24.044072 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:24.044283 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:24.044925 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:24.044930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:24.044934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:24.045427 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:24.045611 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.046405 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.046953 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2145,"thread_start_us":2027,"threads_started":1}
I20260807 04:09:24.049012 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.022455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41079 (local address 127.18.193.3:38773)
0807 04:09:24.023292 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.023343 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.025522 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.026361 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.026395 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.027280 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:24.028336 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.028363 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.030243 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.030251 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.032594 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.032618 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.033820 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.033828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.034061 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.035024 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:24.035064 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:24.038149 (+ 3085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:24.043362 (+ 5213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:24.043378 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:24.043401 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:24.043893 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:24.044446 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:24.044450 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:24.044452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:24.044775 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:24.045580 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:24.045591 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:24.045996 (+ 405us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.046284 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.047207 (+ 923us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":349,"threads_started":1}
I20260807 04:09:24.075065 20772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.056060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:45983 (local address 127.18.193.62:41079)
0807 04:09:24.056315 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.056325 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.058152 (+ 1827us) server_negotiation.cc:407] Connection header received
0807 04:09:24.058299 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.058308 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.058433 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.058571 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:24.059580 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.061058 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.063196 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.064135 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.064306 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.067049 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:24.067056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:24.067060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:24.067103 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:24.071410 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:24.072454 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:24.072463 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:24.072469 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:24.072672 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:24.073247 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:24.073255 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:24.073261 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:24.073700 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:24.073887 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.074627 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.074815 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:09:24.075695 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.056040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41079 (local address 127.18.193.3:45983)
0807 04:09:24.058008 (+ 1968us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.058028 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.058182 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.058637 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.058640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.059118 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:24.059420 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.059430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.061254 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.061262 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.062969 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.062992 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.064378 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.064387 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.064665 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.065485 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:24.065508 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:24.066802 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:24.071569 (+ 4767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:24.071575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:24.071580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:24.072292 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:24.072842 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:24.072848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:24.072852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:24.073074 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:24.073992 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:24.074000 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:24.074290 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.074498 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.074977 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1778,"thread_start_us":1621,"threads_started":1}
I20260807 04:09:24.079123 20746 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:41283
I20260807 04:09:24.079159 20887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:41283 every 8 connection(s)
I20260807 04:09:24.080334 20746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:24.082373 20888 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41079
I20260807 04:09:24.082628 20888 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:24.083253 20888 heartbeater.cc:511] Master 127.18.193.62:41079 requested a full tablet report, sending...
I20260807 04:09:24.084352 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 20746
I20260807 04:09:24.084558 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:24.084797 20381 ts_manager.cc:194] Registered new tserver with Master: 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283)
I20260807 04:09:24.085139 19204 external_mini_cluster.cc:1428] Setting key 665e680620bb81f1b163630a9da5982c
I20260807 04:09:24.085750 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:38773
I20260807 04:09:24.088263 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:09:24 dist-test-slave-8n49 krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:09:24 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:24.139772 20772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.125041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.18.193.62:41079)
0807 04:09:24.125310 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.125318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.125613 (+ 295us) server_negotiation.cc:407] Connection header received
0807 04:09:24.125853 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.125859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.125941 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.126056 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:24.127531 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.128278 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.129672 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.130094 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.130254 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.134311 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:24.134316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:24.134319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:24.134355 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:24.137024 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:24.137705 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:24.137710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:24.137712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:24.137843 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:24.138261 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:24.138265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:24.138267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:24.138563 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:24.138870 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.139167 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.139396 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 04:09:24.144522 20380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:24.151940 20380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:24.180774 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.165172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:34901 (local address 127.0.0.1:41882)
0807 04:09:24.166017 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.166054 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.166264 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.167702 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.167714 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.167757 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:24.168319 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.168332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.172767 (+ 4435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.172777 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.176366 (+ 3589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.176391 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.179364 (+ 2973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.179374 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.179699 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.179786 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.179939 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.180227 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":313,"threads_started":1}
I20260807 04:09:24.182687 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.165303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.18.193.1:34901)
0807 04:09:24.166913 (+ 1610us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.166925 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.166961 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:09:24.167096 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.167107 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.167412 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.168580 (+ 1168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:24.168858 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.172529 (+ 3671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.176760 (+ 4231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.179029 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.181555 (+ 2526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.181880 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.181983 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.182246 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1385,"thread_start_us":1226,"threads_started":1}
I20260807 04:09:24.188721 20525 tablet_service.cc:1467] Processing CreateTablet for tablet fdde795c36274db386d42fc5b64408ff (DEFAULT_TABLE table=test-table [id=28905040cecd41f9bf1d7d73c5971bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:24.190300 20525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdde795c36274db386d42fc5b64408ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:24.193002 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.166329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44309 (local address 127.0.0.1:60332)
0807 04:09:24.181143 (+ 14814us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.181170 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.181330 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.185882 (+ 4552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.185886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.185911 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:24.186239 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.186253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.188132 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.188138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.190207 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.190229 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.192101 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.192107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.192272 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.192304 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.192413 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.192607 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14659,"mutex_wait_us":89,"thread_start_us":18304,"threads_started":1}
I20260807 04:09:24.193013 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.166659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.18.193.2:44309)
0807 04:09:24.185261 (+ 18602us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.185276 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.185298 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:09:24.185383 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.185388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.185637 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.186082 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:24.186402 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.187957 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.190389 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.191899 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.192054 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.192254 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.192504 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.192704 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18448,"thread_start_us":18321,"threads_started":1}
I20260807 04:09:24.193709 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.167564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:41283 (local address 127.0.0.1:58000)
0807 04:09:24.184465 (+ 16901us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.184492 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.184756 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.185544 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.185548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.185573 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:24.186025 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.186039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.188742 (+ 2703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.188747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.190592 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.190607 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.192779 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.192785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.192993 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.193031 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.193229 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.193342 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16694,"mutex_wait_us":24,"thread_start_us":16530,"threads_started":1}
I20260807 04:09:24.194492 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.168314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.18.193.3:41283)
0807 04:09:24.183895 (+ 15581us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.183908 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.184643 (+ 735us) server_negotiation.cc:407] Connection header received
0807 04:09:24.184969 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.184981 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.185314 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.185530 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:24.186416 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.188551 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.190790 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.192214 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.192342 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.192632 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.194073 (+ 1441us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.194201 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15336,"thread_start_us":15029,"threads_started":1}
I20260807 04:09:24.197386 20669 tablet_service.cc:1467] Processing CreateTablet for tablet fdde795c36274db386d42fc5b64408ff (DEFAULT_TABLE table=test-table [id=28905040cecd41f9bf1d7d73c5971bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:24.197841 20816 tablet_service.cc:1467] Processing CreateTablet for tablet fdde795c36274db386d42fc5b64408ff (DEFAULT_TABLE table=test-table [id=28905040cecd41f9bf1d7d73c5971bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:24.199004 20669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdde795c36274db386d42fc5b64408ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:24.199165 20816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdde795c36274db386d42fc5b64408ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:24.203579 20908 tablet_bootstrap.cc:492] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Bootstrap starting.
I20260807 04:09:24.207896 20908 tablet_bootstrap.cc:654] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:24.209059 20908 log.cc:824] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:24.216761 20908 tablet_bootstrap.cc:492] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: No bootstrap required, opened a new log
I20260807 04:09:24.217204 20908 ts_tablet_manager.cc:1397] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260807 04:09:24.217252 20911 tablet_bootstrap.cc:492] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Bootstrap starting.
I20260807 04:09:24.217552 20910 tablet_bootstrap.cc:492] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Bootstrap starting.
I20260807 04:09:24.221237 20910 tablet_bootstrap.cc:654] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:24.222532 20910 log.cc:824] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:24.224164 20911 tablet_bootstrap.cc:654] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:24.225251 20911 log.cc:824] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:24.225911 20910 tablet_bootstrap.cc:492] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: No bootstrap required, opened a new log
I20260807 04:09:24.226197 20910 ts_tablet_manager.cc:1397] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 04:09:24.228068 20911 tablet_bootstrap.cc:492] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: No bootstrap required, opened a new log
I20260807 04:09:24.227906 20908 raft_consensus.cc:348] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.228507 20911 ts_tablet_manager.cc:1397] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260807 04:09:24.228544 20908 raft_consensus.cc:374] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:24.228785 20908 raft_consensus.cc:729] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ddb7dcf9b5d4f0082a262b020c1b75b, State: Initialized, Role: FOLLOWER
I20260807 04:09:24.229465 20908 consensus_queue.cc:260] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.231269 20908 ts_tablet_manager.cc:1428] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 04:09:24.237095 20910 raft_consensus.cc:348] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.237725 20910 raft_consensus.cc:374] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:24.237849 20910 raft_consensus.cc:729] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc2e8bd4cd744e7ab647afb84990b27, State: Initialized, Role: FOLLOWER
I20260807 04:09:24.237636 20911 raft_consensus.cc:348] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.238077 20911 raft_consensus.cc:374] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:24.238215 20911 raft_consensus.cc:729] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f89593f01a4c25bad0de7b5e7ad6cb, State: Initialized, Role: FOLLOWER
I20260807 04:09:24.238520 20910 consensus_queue.cc:260] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.238767 20911 consensus_queue.cc:260] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.239665 20742 heartbeater.cc:503] Master 127.18.193.62:41079 was elected leader, sending a full tablet report...
I20260807 04:09:24.240227 20911 ts_tablet_manager.cc:1428] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260807 04:09:24.240367 20910 ts_tablet_manager.cc:1428] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 04:09:24.240700 20888 heartbeater.cc:503] Master 127.18.193.62:41079 was elected leader, sending a full tablet report...
W20260807 04:09:24.247649 20743 tablet.cc:2369] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:09:24.332180 20889 tablet.cc:2369] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:09:24.354305 20915 raft_consensus.cc:482] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:24.354684 20915 raft_consensus.cc:504] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.356716 20915 leader_election.cc:288] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ddb7dcf9b5d4f0082a262b020c1b75b (127.18.193.1:34901), 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283)
I20260807 04:09:24.364210 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.357260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:47101 (local address 127.18.193.3:41283)
0807 04:09:24.357512 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.357522 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.358656 (+ 1134us) server_negotiation.cc:407] Connection header received
0807 04:09:24.358807 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.358811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.358908 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.359006 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:24.359668 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.360458 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.362451 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.363238 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.363418 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.363499 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.363822 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.363947 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:09:24.364355 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.356930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:34901 (local address 127.18.193.2:32917)
0807 04:09:24.358346 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.358383 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.358535 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.358917 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.358922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.358950 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:24.359357 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.359369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.360425 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.360431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.362047 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.362060 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.363356 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.363363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.363570 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.363613 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.363688 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.363845 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1256,"mutex_wait_us":6,"thread_start_us":1260,"threads_started":1}
I20260807 04:09:24.364399 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.357267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:32917 (local address 127.18.193.1:34901)
0807 04:09:24.357740 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.357749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.358504 (+ 755us) server_negotiation.cc:407] Connection header received
0807 04:09:24.358688 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.358693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.358783 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.358864 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:24.359528 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.360287 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.362214 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.363127 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.363261 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.363358 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.363752 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.364108 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287}
I20260807 04:09:24.364398 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.357168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:41283 (local address 127.18.193.2:47101)
0807 04:09:24.358527 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.358545 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.358680 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.359056 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.359063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.359093 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:24.359485 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.359498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.360707 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.360712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.362278 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.362294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.363476 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.363481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.363627 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.363651 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.363772 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.363919 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"mutex_wait_us":26,"thread_start_us":798,"threads_started":1}
I20260807 04:09:24.365701 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdde795c36274db386d42fc5b64408ff" candidate_uuid: "adc2e8bd4cd744e7ab647afb84990b27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" is_pre_election: true
I20260807 04:09:24.365739 20547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdde795c36274db386d42fc5b64408ff" candidate_uuid: "adc2e8bd4cd744e7ab647afb84990b27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" is_pre_election: true
I20260807 04:09:24.366237 20841 raft_consensus.cc:2454] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc2e8bd4cd744e7ab647afb84990b27 in term 0.
I20260807 04:09:24.366251 20547 raft_consensus.cc:2454] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc2e8bd4cd744e7ab647afb84990b27 in term 0.
I20260807 04:09:24.366963 20619 leader_election.cc:302] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ddb7dcf9b5d4f0082a262b020c1b75b, 92f89593f01a4c25bad0de7b5e7ad6cb, adc2e8bd4cd744e7ab647afb84990b27; no voters:
I20260807 04:09:24.367369 20915 raft_consensus.cc:2781] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:24.367535 20915 raft_consensus.cc:482] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:24.367647 20915 raft_consensus.cc:3037] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:24.371321 20915 raft_consensus.cc:504] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.372270 20915 leader_election.cc:288] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [CANDIDATE]: Term 1 election: Requested vote from peers 0ddb7dcf9b5d4f0082a262b020c1b75b (127.18.193.1:34901), 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283)
I20260807 04:09:24.372831 20547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdde795c36274db386d42fc5b64408ff" candidate_uuid: "adc2e8bd4cd744e7ab647afb84990b27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b"
I20260807 04:09:24.372889 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdde795c36274db386d42fc5b64408ff" candidate_uuid: "adc2e8bd4cd744e7ab647afb84990b27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb"
I20260807 04:09:24.373080 20547 raft_consensus.cc:3037] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:24.373131 20841 raft_consensus.cc:3037] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:24.376897 20547 raft_consensus.cc:2454] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc2e8bd4cd744e7ab647afb84990b27 in term 1.
I20260807 04:09:24.376896 20841 raft_consensus.cc:2454] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc2e8bd4cd744e7ab647afb84990b27 in term 1.
I20260807 04:09:24.377467 20620 leader_election.cc:302] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ddb7dcf9b5d4f0082a262b020c1b75b, adc2e8bd4cd744e7ab647afb84990b27; no voters:
I20260807 04:09:24.377828 20915 raft_consensus.cc:2781] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:24.378473 20915 raft_consensus.cc:686] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 1 LEADER]: Becoming Leader. State: Replica: adc2e8bd4cd744e7ab647afb84990b27, State: Running, Role: LEADER
I20260807 04:09:24.379528 20915 consensus_queue.cc:237] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.386216 20380 catalog_manager.cc:5495] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 reported cstate change: term changed from 0 to 1, leader changed from <none> to adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2). New cstate: current_term: 1 leader_uuid: "adc2e8bd4cd744e7ab647afb84990b27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } health_report { overall_health: UNKNOWN } } }
I20260807 04:09:24.423242 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.417270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60346 (local address 127.18.193.2:44309)
0807 04:09:24.417506 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.417548 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.417668 (+ 120us) server_negotiation.cc:407] Connection header received
0807 04:09:24.418012 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.418018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.418119 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.418284 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:24.418650 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.419406 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.420869 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.421318 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.421478 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.422015 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:24.422598 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:24.422747 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.422876 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.422961 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:09:24.431742 20547 raft_consensus.cc:1261] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Refusing update from remote peer adc2e8bd4cd744e7ab647afb84990b27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:24.431752 20841 raft_consensus.cc:1261] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Refusing update from remote peer adc2e8bd4cd744e7ab647afb84990b27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:24.432838 20919 consensus_queue.cc:1035] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:24.433236 20915 consensus_queue.cc:1035] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:24.447511 20924 mvcc.cc:205] Tried to move back new op lower bound from 7315766331102617600 to 7315766330904154112. Current Snapshot: MvccSnapshot[applied={T|T < 7315766331102617600}]
I20260807 04:09:24.465644 20380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:24.465977 20380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:24.470235 20380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e: Sending DeleteTablet for 3 replicas of tablet fdde795c36274db386d42fc5b64408ff
I20260807 04:09:24.471554 20525 tablet_service.cc:1514] Processing DeleteTablet for tablet fdde795c36274db386d42fc5b64408ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:24 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:41882
I20260807 04:09:24.471678 20669 tablet_service.cc:1514] Processing DeleteTablet for tablet fdde795c36274db386d42fc5b64408ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:24 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:60332
I20260807 04:09:24.472071 20816 tablet_service.cc:1514] Processing DeleteTablet for tablet fdde795c36274db386d42fc5b64408ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:24 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:58000
I20260807 04:09:24.474879 20380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:24.475881 20380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:24.486330 20932 tablet_replica.cc:330] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: stopping tablet replica
I20260807 04:09:24.487028 20932 raft_consensus.cc:2229] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:09:24.488022 20932 raft_consensus.cc:2258] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:24.489050 20931 tablet_replica.cc:330] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: stopping tablet replica
I20260807 04:09:24.489663 20931 raft_consensus.cc:2229] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:24.490355 20931 raft_consensus.cc:2258] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:24.491053 20815 tablet_service.cc:1467] Processing CreateTablet for tablet b84655ef274e45418255f64daf453962 (DEFAULT_TABLE table=test-table [id=baa6d3ee511142bf897f67cccbb6898f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:24.491279 20933 tablet_replica.cc:330] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: stopping tablet replica
I20260807 04:09:24.491714 20815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84655ef274e45418255f64daf453962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:24.491789 20933 raft_consensus.cc:2229] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:24.493063 20933 raft_consensus.cc:2258] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:24.493176 20669 tablet_service.cc:1467] Processing CreateTablet for tablet b84655ef274e45418255f64daf453962 (DEFAULT_TABLE table=test-table [id=baa6d3ee511142bf897f67cccbb6898f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:24.493664 20669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84655ef274e45418255f64daf453962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:24.495272 20931 ts_tablet_manager.cc:1905] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:24.498805 20911 tablet_bootstrap.cc:492] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: Bootstrap starting.
I20260807 04:09:24.499811 20933 ts_tablet_manager.cc:1905] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:24.500316 20932 ts_tablet_manager.cc:1905] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:24.500497 20911 tablet_bootstrap.cc:654] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:24.501014 20525 tablet_service.cc:1467] Processing CreateTablet for tablet b84655ef274e45418255f64daf453962 (DEFAULT_TABLE table=test-table [id=baa6d3ee511142bf897f67cccbb6898f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:24.506997 20525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84655ef274e45418255f64daf453962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:24.508962 20933 ts_tablet_manager.cc:1918] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:24.509140 20911 tablet_bootstrap.cc:492] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: No bootstrap required, opened a new log
I20260807 04:09:24.509316 20911 ts_tablet_manager.cc:1397] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260807 04:09:24.510151 20911 raft_consensus.cc:348] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.510396 20911 raft_consensus.cc:374] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:24.510489 20911 raft_consensus.cc:729] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f89593f01a4c25bad0de7b5e7ad6cb, State: Initialized, Role: FOLLOWER
I20260807 04:09:24.510805 20911 consensus_queue.cc:260] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.511515 20911 ts_tablet_manager.cc:1428] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:09:24.513433 20931 ts_tablet_manager.cc:1918] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:24.513631 20931 log.cc:1196] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/fdde795c36274db386d42fc5b64408ff
I20260807 04:09:24.514249 20931 ts_tablet_manager.cc:1939] T fdde795c36274db386d42fc5b64408ff P 0ddb7dcf9b5d4f0082a262b020c1b75b: Deleting consensus metadata
I20260807 04:09:24.515566 20910 tablet_bootstrap.cc:492] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: Bootstrap starting.
I20260807 04:09:24.516304 20366 catalog_manager.cc:4841] TS 0ddb7dcf9b5d4f0082a262b020c1b75b (127.18.193.1:34901): tablet fdde795c36274db386d42fc5b64408ff (table test-table [id=28905040cecd41f9bf1d7d73c5971bbc]) successfully deleted
I20260807 04:09:24.509160 20933 log.cc:1196] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/fdde795c36274db386d42fc5b64408ff
I20260807 04:09:24.517498 20910 tablet_bootstrap.cc:654] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:24.517891 20933 ts_tablet_manager.cc:1939] T fdde795c36274db386d42fc5b64408ff P 92f89593f01a4c25bad0de7b5e7ad6cb: Deleting consensus metadata
I20260807 04:09:24.519622 20908 tablet_bootstrap.cc:492] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Bootstrap starting.
I20260807 04:09:24.521905 20908 tablet_bootstrap.cc:654] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:24.523361 20932 ts_tablet_manager.cc:1918] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:24.523559 20932 log.cc:1196] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/fdde795c36274db386d42fc5b64408ff
I20260807 04:09:24.524165 20932 ts_tablet_manager.cc:1939] T fdde795c36274db386d42fc5b64408ff P adc2e8bd4cd744e7ab647afb84990b27: Deleting consensus metadata
I20260807 04:09:24.526500 20365 catalog_manager.cc:4841] TS 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283): tablet fdde795c36274db386d42fc5b64408ff (table test-table [id=28905040cecd41f9bf1d7d73c5971bbc]) successfully deleted
I20260807 04:09:24.527649 20908 tablet_bootstrap.cc:492] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: No bootstrap required, opened a new log
I20260807 04:09:24.527820 20908 ts_tablet_manager.cc:1397] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 04:09:24.528934 20366 catalog_manager.cc:4841] TS adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2:44309): tablet fdde795c36274db386d42fc5b64408ff (table test-table [id=28905040cecd41f9bf1d7d73c5971bbc]) successfully deleted
I20260807 04:09:24.528779 20908 raft_consensus.cc:348] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.529376 20908 raft_consensus.cc:374] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:24.529486 20908 raft_consensus.cc:729] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ddb7dcf9b5d4f0082a262b020c1b75b, State: Initialized, Role: FOLLOWER
I20260807 04:09:24.529815 20908 consensus_queue.cc:260] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.530437 20908 ts_tablet_manager.cc:1428] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 04:09:24.530799 20910 tablet_bootstrap.cc:492] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: No bootstrap required, opened a new log
I20260807 04:09:24.531019 20910 ts_tablet_manager.cc:1397] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20260807 04:09:24.531723 20910 raft_consensus.cc:348] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.531927 20910 raft_consensus.cc:374] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:24.532029 20910 raft_consensus.cc:729] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc2e8bd4cd744e7ab647afb84990b27, State: Initialized, Role: FOLLOWER
I20260807 04:09:24.532331 20910 consensus_queue.cc:260] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.534915 20910 ts_tablet_manager.cc:1428] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260807 04:09:24.704962 20598 tablet.cc:2369] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:09:24.843791 20914 raft_consensus.cc:482] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:24.844075 20914 raft_consensus.cc:504] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.845852 20914 leader_election.cc:288] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2:44309), 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283)
I20260807 04:09:24.859647 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.846736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44309 (local address 127.18.193.1:33359)
0807 04:09:24.848201 (+ 1465us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.848220 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.848370 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.848984 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.848990 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.849014 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:24.849813 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.849823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.851139 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.851145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.855407 (+ 4262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.855416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.858088 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.858095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.858322 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.858383 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.858662 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.858852 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1289,"mutex_wait_us":121,"thread_start_us":946,"threads_started":1}
I20260807 04:09:24.859675 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.847180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:33359 (local address 127.18.193.2:44309)
0807 04:09:24.847461 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.847468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.848340 (+ 872us) server_negotiation.cc:407] Connection header received
0807 04:09:24.848579 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.848586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.848741 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.848904 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:24.850003 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.851001 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.855667 (+ 4666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.857616 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.857843 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.857976 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.858735 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.858977 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 04:09:24.859753 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.846344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:37001 (local address 127.18.193.3:41283)
0807 04:09:24.846555 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.846565 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.847239 (+ 674us) server_negotiation.cc:407] Connection header received
0807 04:09:24.847967 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.847988 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.848074 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.848160 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:24.849369 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.851208 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.855660 (+ 4452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.858147 (+ 2487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.858417 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.858500 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:24.859179 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 04:09:24.859347 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:09:24.860685 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.846299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:41283 (local address 127.18.193.1:37001)
0807 04:09:24.847113 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.847137 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.847664 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.848425 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.848431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.848459 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:24.849004 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.849015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.851433 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.851438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.855380 (+ 3942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.855401 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.858625 (+ 3224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.858636 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.858889 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:24.858920 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:09:24.859996 (+ 1076us) client_negotiation.cc:240] Negotiation successful
0807 04:09:24.860106 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":70,"thread_start_us":453,"threads_started":1}
I20260807 04:09:24.861997 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84655ef274e45418255f64daf453962" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" is_pre_election: true
I20260807 04:09:24.861657 20690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84655ef274e45418255f64daf453962" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc2e8bd4cd744e7ab647afb84990b27" is_pre_election: true
I20260807 04:09:24.862458 20841 raft_consensus.cc:2454] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 0.
I20260807 04:09:24.862632 20690 raft_consensus.cc:2454] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 0.
I20260807 04:09:24.863186 20474 leader_election.cc:302] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ddb7dcf9b5d4f0082a262b020c1b75b, 92f89593f01a4c25bad0de7b5e7ad6cb; no voters:
I20260807 04:09:24.863936 20914 raft_consensus.cc:2781] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:24.864218 20914 raft_consensus.cc:482] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:24.864390 20914 raft_consensus.cc:3037] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:24.867623 20914 raft_consensus.cc:504] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.868345 20914 leader_election.cc:288] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 election: Requested vote from peers adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2:44309), 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283)
I20260807 04:09:24.869020 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84655ef274e45418255f64daf453962" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb"
I20260807 04:09:24.869251 20841 raft_consensus.cc:3037] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:24.869405 20690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84655ef274e45418255f64daf453962" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc2e8bd4cd744e7ab647afb84990b27"
I20260807 04:09:24.869715 20690 raft_consensus.cc:3037] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:24.871487 20841 raft_consensus.cc:2454] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 1.
I20260807 04:09:24.871959 20474 leader_election.cc:302] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ddb7dcf9b5d4f0082a262b020c1b75b, 92f89593f01a4c25bad0de7b5e7ad6cb; no voters:
I20260807 04:09:24.872316 20914 raft_consensus.cc:2781] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:24.873286 20914 raft_consensus.cc:686] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 LEADER]: Becoming Leader. State: Replica: 0ddb7dcf9b5d4f0082a262b020c1b75b, State: Running, Role: LEADER
I20260807 04:09:24.874092 20914 consensus_queue.cc:237] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:24.874732 20690 raft_consensus.cc:2454] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 1.
I20260807 04:09:24.878404 20381 catalog_manager.cc:5495] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ddb7dcf9b5d4f0082a262b020c1b75b (127.18.193.1). New cstate: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } health_report { overall_health: UNKNOWN } } }
Aug 07 04:09:25 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075762, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:25.010829 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.992266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41079 (local address 127.0.0.1:33500)
0807 04:09:24.993536 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:24.993555 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:24.993717 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:24.994055 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:24.994059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:24.994950 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:24.995328 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.995338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.996926 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.996931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.998374 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:24.998388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.999500 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.999506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:24.999654 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.000492 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:25.000517 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:25.003763 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:25.007535 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:25.007550 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:25.007559 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:25.008133 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:25.008588 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:25.008593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:25.008599 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:25.008825 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:25.009394 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:25.009402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:25.009628 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 04:09:25.009808 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 04:09:25.010107 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"thread_start_us":851,"threads_started":1}
I20260807 04:09:25.010829 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:24.991548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33500 (local address 127.18.193.62:41079)
0807 04:09:24.992589 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:24.992598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:24.993656 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 04:09:24.993828 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:24.993832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:24.993892 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:24.994005 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:24.995491 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.996785 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.998564 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:24.999290 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:24.999388 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.004081 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:25.004087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:25.004091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:25.004121 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:25.007374 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.008258 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.008276 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.008279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.008443 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.008946 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.008951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.008954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.009260 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:25.009419 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.009902 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.010247 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":834,"thread_start_us":638,"threads_started":1}
I20260807 04:09:25.017093 20381 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:33500:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 04:09:25.018358 20381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:25.025864 20669 tablet_service.cc:1467] Processing CreateTablet for tablet 039c612dce12411ba9710d94fb80bd1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75893640f564e83ac6621c417081858]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:09:25.025853 20525 tablet_service.cc:1467] Processing CreateTablet for tablet 039c612dce12411ba9710d94fb80bd1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75893640f564e83ac6621c417081858]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:09:25.026516 20525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039c612dce12411ba9710d94fb80bd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:25.026539 20669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039c612dce12411ba9710d94fb80bd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:25.026341 20815 tablet_service.cc:1467] Processing CreateTablet for tablet 039c612dce12411ba9710d94fb80bd1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75893640f564e83ac6621c417081858]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:09:25.027087 20815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039c612dce12411ba9710d94fb80bd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:25.034557 20948 tablet_bootstrap.cc:492] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: Bootstrap starting.
I20260807 04:09:25.035490 20910 tablet_bootstrap.cc:492] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: Bootstrap starting.
I20260807 04:09:25.036978 20948 tablet_bootstrap.cc:654] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:25.038720 20910 tablet_bootstrap.cc:654] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:25.042354 20949 tablet_bootstrap.cc:492] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: Bootstrap starting.
I20260807 04:09:25.043767 20910 tablet_bootstrap.cc:492] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: No bootstrap required, opened a new log
I20260807 04:09:25.044150 20910 ts_tablet_manager.cc:1397] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20260807 04:09:25.045073 20948 tablet_bootstrap.cc:492] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: No bootstrap required, opened a new log
I20260807 04:09:25.045260 20949 tablet_bootstrap.cc:654] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:25.045357 20948 ts_tablet_manager.cc:1397] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260807 04:09:25.045574 20910 raft_consensus.cc:348] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.046046 20910 raft_consensus.cc:374] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:25.046243 20910 raft_consensus.cc:729] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc2e8bd4cd744e7ab647afb84990b27, State: Initialized, Role: FOLLOWER
I20260807 04:09:25.046689 20948 raft_consensus.cc:348] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.046705 20910 consensus_queue.cc:260] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.047175 20948 raft_consensus.cc:374] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:25.047412 20948 raft_consensus.cc:729] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ddb7dcf9b5d4f0082a262b020c1b75b, State: Initialized, Role: FOLLOWER
I20260807 04:09:25.047830 20948 consensus_queue.cc:260] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.048506 20914 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.049022 20914 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:25.050661 20948 ts_tablet_manager.cc:1428] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260807 04:09:25.052071 20910 ts_tablet_manager.cc:1428] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
I20260807 04:09:25.053755 20953 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.054265 20953 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:25.055055 20949 tablet_bootstrap.cc:492] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: No bootstrap required, opened a new log
I20260807 04:09:25.055313 20949 ts_tablet_manager.cc:1397] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260807 04:09:25.056389 20949 raft_consensus.cc:348] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.056639 20949 raft_consensus.cc:374] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:25.056746 20949 raft_consensus.cc:729] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f89593f01a4c25bad0de7b5e7ad6cb, State: Initialized, Role: FOLLOWER
I20260807 04:09:25.056998 20949 consensus_queue.cc:260] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.058183 20949 ts_tablet_manager.cc:1428] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 04:09:25.058233 20954 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.058800 20954 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:25.194571 20914 raft_consensus.cc:482] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:25.194785 20914 raft_consensus.cc:504] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.195437 20914 leader_election.cc:288] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2:44309), 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283)
I20260807 04:09:25.196151 20690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039c612dce12411ba9710d94fb80bd1f" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc2e8bd4cd744e7ab647afb84990b27" is_pre_election: true
I20260807 04:09:25.196153 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039c612dce12411ba9710d94fb80bd1f" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" is_pre_election: true
I20260807 04:09:25.196486 20841 raft_consensus.cc:2454] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 0.
I20260807 04:09:25.196538 20690 raft_consensus.cc:2454] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 0.
I20260807 04:09:25.197062 20474 leader_election.cc:302] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ddb7dcf9b5d4f0082a262b020c1b75b, 92f89593f01a4c25bad0de7b5e7ad6cb; no voters:
I20260807 04:09:25.197501 20914 raft_consensus.cc:2781] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:25.197664 20914 raft_consensus.cc:482] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:25.197891 20914 raft_consensus.cc:3037] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:25.200309 20914 raft_consensus.cc:504] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.200968 20914 leader_election.cc:288] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 election: Requested vote from peers adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2:44309), 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283)
I20260807 04:09:25.201445 20690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039c612dce12411ba9710d94fb80bd1f" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc2e8bd4cd744e7ab647afb84990b27"
I20260807 04:09:25.201503 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039c612dce12411ba9710d94fb80bd1f" candidate_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb"
I20260807 04:09:25.201678 20690 raft_consensus.cc:3037] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:25.201746 20841 raft_consensus.cc:3037] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:25.203845 20690 raft_consensus.cc:2454] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 1.
I20260807 04:09:25.204119 20841 raft_consensus.cc:2454] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ddb7dcf9b5d4f0082a262b020c1b75b in term 1.
I20260807 04:09:25.204355 20475 leader_election.cc:302] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ddb7dcf9b5d4f0082a262b020c1b75b, adc2e8bd4cd744e7ab647afb84990b27; no voters:
I20260807 04:09:25.204715 20914 raft_consensus.cc:2781] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:25.204936 20914 raft_consensus.cc:686] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 LEADER]: Becoming Leader. State: Replica: 0ddb7dcf9b5d4f0082a262b020c1b75b, State: Running, Role: LEADER
I20260807 04:09:25.205192 20914 consensus_queue.cc:237] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } }
I20260807 04:09:25.207029 20939 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: TxnStatusTablet state changed. Reason: New leader 0ddb7dcf9b5d4f0082a262b020c1b75b. Latest consensus state: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.207369 20939 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:09:25.209064 20379 catalog_manager.cc:5495] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ddb7dcf9b5d4f0082a262b020c1b75b (127.18.193.1). New cstate: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } health_report { overall_health: UNKNOWN } } }
I20260807 04:09:25.213358 20956 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 04:09:25.213681 20956 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 04:09:25.240377 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.232731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:34901 (local address 127.0.0.1:41884)
0807 04:09:25.233059 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:25.233087 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:25.233248 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:25.234154 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:25.234157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:25.234168 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:25.234336 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:25.234343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.236228 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.236232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:25.237345 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:25.237358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.238484 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.238489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:25.238670 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.238732 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:25.239906 (+ 1174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:25.239927 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 04:09:25.240000 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 04:09:25.240137 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 04:09:25.240545 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.232765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.18.193.1:34901)
0807 04:09:25.233607 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.233613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.233642 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:09:25.233739 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.233745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.233911 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.234159 (+ 248us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:25.234625 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.236046 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.237500 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.238272 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.238411 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.238966 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:25.239786 (+ 820us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:25.239935 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.240047 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.240135 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":349,"threads_started":1}
I20260807 04:09:25.247233 20841 raft_consensus.cc:1261] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Refusing update from remote peer 0ddb7dcf9b5d4f0082a262b020c1b75b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:25.247570 20690 raft_consensus.cc:1261] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Refusing update from remote peer 0ddb7dcf9b5d4f0082a262b020c1b75b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:25.248694 20939 consensus_queue.cc:1035] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [LEADER]: Connected to new peer: Peer: permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:25.250123 20939 consensus_queue.cc:1035] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:25.268802 20954 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: TxnStatusTablet state changed. Reason: New leader 0ddb7dcf9b5d4f0082a262b020c1b75b. Latest consensus state: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.269320 20954 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:25.272511 20953 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: TxnStatusTablet state changed. Reason: New leader 0ddb7dcf9b5d4f0082a262b020c1b75b. Latest consensus state: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.273049 20953 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:25.274788 20939 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.275156 20939 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:09:25.279616 20961 mvcc.cc:205] Tried to move back new op lower bound from 7315766334442246144 to 7315766334283231232. Current Snapshot: MvccSnapshot[applied={T|T < 7315766334442246144}]
I20260807 04:09:25.292883 20914 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.293447 20914 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P 0ddb7dcf9b5d4f0082a262b020c1b75b: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:09:25.296902 20954 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.297205 20954 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P 92f89593f01a4c25bad0de7b5e7ad6cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:25.301180 20953 tablet_replica.cc:439] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ddb7dcf9b5d4f0082a262b020c1b75b" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34901 } } peers { permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 } } peers { permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 } } }
I20260807 04:09:25.301466 20953 tablet_replica.cc:442] T 039c612dce12411ba9710d94fb80bd1f P adc2e8bd4cd744e7ab647afb84990b27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:25.309078 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.300906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.18.193.1:34901)
0807 04:09:25.301244 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.301254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.303391 (+ 2137us) server_negotiation.cc:407] Connection header received
0807 04:09:25.303517 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.303523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.303621 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.303732 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:25.304154 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.305418 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.307090 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.307811 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.307943 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.308320 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:25.308476 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:25.308600 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.308723 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.308803 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260807 04:09:25.330494 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.322209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:37347 (local address 127.18.193.1:34901)
0807 04:09:25.322442 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.322447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.323642 (+ 1195us) server_negotiation.cc:407] Connection header received
0807 04:09:25.323897 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.323902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.324044 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.324163 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:25.324981 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.326065 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.328067 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.328763 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.328916 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.329432 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:25.329538 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:25.329652 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.329786 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.329865 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 04:09:25.330586 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.322272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:34901 (local address 127.18.193.1:37347)
0807 04:09:25.323517 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:25.323534 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:25.323764 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:25.324175 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:25.324179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:25.324189 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:25.324798 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:25.324810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.326226 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.326234 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:25.327904 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:25.327917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.329038 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.329044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:25.329156 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.329220 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:25.329651 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:25.329667 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 04:09:25.329770 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 04:09:25.329917 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":726,"threads_started":1}
I20260807 04:09:25.342962 20841 raft_consensus.cc:1261] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Refusing update from remote peer 0ddb7dcf9b5d4f0082a262b020c1b75b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:25.343132 20690 raft_consensus.cc:1261] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Refusing update from remote peer 0ddb7dcf9b5d4f0082a262b020c1b75b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:25.343833 20965 consensus_queue.cc:1035] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f89593f01a4c25bad0de7b5e7ad6cb" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 41283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:25.344260 20939 consensus_queue.cc:1035] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [LEADER]: Connected to new peer: Peer: permanent_uuid: "adc2e8bd4cd744e7ab647afb84990b27" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:25.427695 20379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:25.428072 20379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:25.432013 20379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f59bb9c46c3a4897abc631bb8ace809e: Sending DeleteTablet for 3 replicas of tablet b84655ef274e45418255f64daf453962
I20260807 04:09:25.433450 20669 tablet_service.cc:1514] Processing DeleteTablet for tablet b84655ef274e45418255f64daf453962 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:25 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:60332
I20260807 04:09:25.433471 20815 tablet_service.cc:1514] Processing DeleteTablet for tablet b84655ef274e45418255f64daf453962 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:25 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:58000
I20260807 04:09:25.434083 20525 tablet_service.cc:1514] Processing DeleteTablet for tablet b84655ef274e45418255f64daf453962 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:25 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:41882
I20260807 04:09:25.446355 20978 tablet_replica.cc:330] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: stopping tablet replica
I20260807 04:09:25.446820 20978 raft_consensus.cc:2229] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:25.447229 20978 raft_consensus.cc:2258] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:25.448791 20978 ts_tablet_manager.cc:1905] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:25.457278 20978 ts_tablet_manager.cc:1918] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:25.457538 20978 log.cc:1196] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/b84655ef274e45418255f64daf453962
I20260807 04:09:25.457803 20979 tablet_replica.cc:330] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: stopping tablet replica
I20260807 04:09:25.458230 20978 ts_tablet_manager.cc:1939] T b84655ef274e45418255f64daf453962 P adc2e8bd4cd744e7ab647afb84990b27: Deleting consensus metadata
I20260807 04:09:25.458199 20979 raft_consensus.cc:2229] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:09:25.459234 20979 raft_consensus.cc:2258] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:25.459640 20366 catalog_manager.cc:4841] TS adc2e8bd4cd744e7ab647afb84990b27 (127.18.193.2:44309): tablet b84655ef274e45418255f64daf453962 (table test-table [id=baa6d3ee511142bf897f67cccbb6898f]) successfully deleted
I20260807 04:09:25.461210 20977 tablet_replica.cc:330] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: stopping tablet replica
I20260807 04:09:25.461395 20979 ts_tablet_manager.cc:1905] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:25.461652 20977 raft_consensus.cc:2229] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:25.462028 20977 raft_consensus.cc:2258] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:25.463403 20977 ts_tablet_manager.cc:1905] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:25.471503 20977 ts_tablet_manager.cc:1918] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:25.471623 20979 ts_tablet_manager.cc:1918] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:25.471673 20977 log.cc:1196] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/b84655ef274e45418255f64daf453962
I20260807 04:09:25.471832 20979 log.cc:1196] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/b84655ef274e45418255f64daf453962
I20260807 04:09:25.472190 20977 ts_tablet_manager.cc:1939] T b84655ef274e45418255f64daf453962 P 92f89593f01a4c25bad0de7b5e7ad6cb: Deleting consensus metadata
I20260807 04:09:25.472503 20979 ts_tablet_manager.cc:1939] T b84655ef274e45418255f64daf453962 P 0ddb7dcf9b5d4f0082a262b020c1b75b: Deleting consensus metadata
I20260807 04:09:25.473519 20365 catalog_manager.cc:4841] TS 92f89593f01a4c25bad0de7b5e7ad6cb (127.18.193.3:41283): tablet b84655ef274e45418255f64daf453962 (table test-table [id=baa6d3ee511142bf897f67cccbb6898f]) successfully deleted
I20260807 04:09:25.473755 20366 catalog_manager.cc:4841] TS 0ddb7dcf9b5d4f0082a262b020c1b75b (127.18.193.1:34901): tablet b84655ef274e45418255f64daf453962 (table test-table [id=baa6d3ee511142bf897f67cccbb6898f]) successfully deleted
Aug 07 04:09:25 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:25.492983 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.463763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.18.193.1:34901)
0807 04:09:25.463983 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.463990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.474582 (+ 10592us) server_negotiation.cc:407] Connection header received
0807 04:09:25.474830 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.474837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.474916 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.475045 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:25.477089 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.478020 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.479398 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.479898 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.480039 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.484678 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:25.484693 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:25.484709 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:25.484744 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:25.488941 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.489826 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.489852 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.489862 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.490139 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.490689 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.490700 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.490703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.491270 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:25.491726 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.492062 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.492414 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 04:09:25.493932 20591 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41904
I20260807 04:09:25.517125 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.497543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.18.193.62:41079)
0807 04:09:25.497931 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.497938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.499022 (+ 1084us) server_negotiation.cc:407] Connection header received
0807 04:09:25.499229 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.499235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.499315 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.499426 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:25.502126 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.503016 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.504897 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.505841 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.505997 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.509336 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:25.509342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:25.509345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:25.509380 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:25.513400 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.514275 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.514281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.514285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.514505 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.514963 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.514974 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.514977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.515411 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:25.515778 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.516194 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.516725 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
W20260807 04:09:25.517987 20379 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33510
I20260807 04:09:25.520576 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20457
I20260807 04:09:25.535413 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.521518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.18.193.1:34901)
0807 04:09:25.521727 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.521734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.522180 (+ 446us) server_negotiation.cc:407] Connection header received
0807 04:09:25.522410 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.522419 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.522517 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.522577 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:25.523836 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.524672 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.525943 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.526487 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.526625 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.529097 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:25.529103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:25.529106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:25.529140 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:25.532359 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.533116 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.533122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.533126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.533303 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.533711 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.533716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.533719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.534049 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:25.534324 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.534665 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.534925 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 04:09:25.536175 20591 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41912
W20260807 04:09:25.536921 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20457: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:25.537159 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 20457
I20260807 04:09:25.560585 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20602
Aug 07 04:09:25 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:09:25.575965 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.561452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.18.193.2:44309)
0807 04:09:25.561875 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.561880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.561896 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:09:25.561959 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.561962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.562020 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.562114 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:25.563359 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.564059 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.565490 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.565925 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.566047 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.570397 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:25.570412 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:25.570429 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:25.570498 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:25.573473 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.574111 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.574118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.574120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.574261 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.574628 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.574632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.574634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.574998 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:25.575221 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.575493 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.575667 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":228,"threads_started":1}
W20260807 04:09:25.576758 20737 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60358
W20260807 04:09:25.577427 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20602: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:25.577616 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 20602
I20260807 04:09:25.602193 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20746
Aug 07 04:09:25 dist-test-slave-8n49 krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:09:25.618129 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:25.603011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.18.193.3:41283)
0807 04:09:25.603536 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:25.603540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:25.603557 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:09:25.603615 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:25.603618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:25.603677 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:25.603768 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:25.605203 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.606046 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.607297 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:25.607705 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:25.607804 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:25.612044 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:25.612068 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:25.612085 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:25.612130 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:25.615289 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.615982 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.615989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.615992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.616129 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:25.616523 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:25.616526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:25.616528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:25.617012 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:25.617249 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:25.617555 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 04:09:25.617725 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":303,"threads_started":1}
W20260807 04:09:25.618862 20883 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58004
W20260807 04:09:25.619611 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20746: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:25.619778 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 20746
I20260807 04:09:25.639263 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 20349
I20260807 04:09:25.640542 20424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:25.758422 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 20349
2026-08-07T04:09:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5420 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:25 dist-test-slave-8n49 krb5kdc[20991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:25 dist-test-slave-8n49 krb5kdc[20991](info): set up 2 sockets
Aug 07 04:09:25 dist-test-slave-8n49 krb5kdc[20991](info): commencing operation
krb5kdc: starting...
W20260807 04:09:26.906646 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:27 dist-test-slave-8n49 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:27.133102 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34981
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:34981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:27.478156 21004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:27.478483 21004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:27.478631 21004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:27.486243 21004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:27.486357 21004 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 04:09:27.486428 21004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:27.486523 21004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:27.486596 21004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:27.486683 21004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:27.497159 21004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:34981
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:27.498530 21004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:27.499877 21004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:27.512744 21013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:27.512844 21010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:27.512841 21011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:27.514026 21004 server_base.cc:1034] running on GCE node
I20260807 04:09:27.514935 21004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:27.516510 21004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:27.517830 21004 hybrid_clock.cc:648] HybridClock initialized: now 1786075767517747 us; error 54 us; skew 500 ppm
I20260807 04:09:27.518229 21004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:27 dist-test-slave-8n49 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075767, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:27.522199 21004 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:27.523828 21004 webserver.cc:460] Webserver started at http://127.18.193.62:40635/ using document root <none> and password file <none>
I20260807 04:09:27.524711 21004 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:27.524845 21004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:27.525225 21004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:27.528601 21004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "a68856f2b0824b31be4e6f485aa4263b"
format_stamp: "Formatted at 2026-08-07 04:09:27 on dist-test-slave-8n49"
server_key: "d154360100d2b5e944d6616892226d54"
server_key_iv: "49498a1a29d9ee9fb78ddeda3a6a57fe"
server_key_version: "encryptionkey@0"
I20260807 04:09:27.529520 21004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "a68856f2b0824b31be4e6f485aa4263b"
format_stamp: "Formatted at 2026-08-07 04:09:27 on dist-test-slave-8n49"
server_key: "d154360100d2b5e944d6616892226d54"
server_key_iv: "49498a1a29d9ee9fb78ddeda3a6a57fe"
server_key_version: "encryptionkey@0"
I20260807 04:09:27.535394 21004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 04:09:27.539433 21020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:27.541051 21004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:27.541275 21004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "a68856f2b0824b31be4e6f485aa4263b"
format_stamp: "Formatted at 2026-08-07 04:09:27 on dist-test-slave-8n49"
server_key: "d154360100d2b5e944d6616892226d54"
server_key_iv: "49498a1a29d9ee9fb78ddeda3a6a57fe"
server_key_version: "encryptionkey@0"
I20260807 04:09:27.541535 21004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:27.557245 21004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:27.562178 21004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:27.562482 21004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:27.586098 21004 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:34981
I20260807 04:09:27.586141 21085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:34981 every 8 connection(s)
I20260807 04:09:27.587551 21004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:27.591450 21086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:27.595176 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 21004
I20260807 04:09:27.595438 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:27.596134 19204 external_mini_cluster.cc:1428] Setting key fb7e1c2b2af89fc36efc4b42b808477e
I20260807 04:09:27.603091 21086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Bootstrap starting.
I20260807 04:09:27.607039 21086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:27.608294 21086 log.cc:824] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:27.611208 21086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: No bootstrap required, opened a new log
Aug 07 04:09:27 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:27.622274 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:27.598333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.18.193.62:34981)
0807 04:09:27.599217 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:27.599228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:27.599271 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:09:27.601403 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:27.601427 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:27.601832 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:27.602183 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:27.604137 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:27.605628 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:27.607633 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:27.608432 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:27.608727 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:27.613026 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:27.613039 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:27.613062 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:27.613125 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:27.617366 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:27.618087 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:27.618097 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:27.618106 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:27.618336 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:27.618735 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:27.618740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:27.618744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:27.619535 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:27.619829 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:27.620286 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 04:09:27.620819 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":393,"threads_started":1}
I20260807 04:09:27.622520 21086 raft_consensus.cc:348] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68856f2b0824b31be4e6f485aa4263b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34981 } }
I20260807 04:09:27.623003 21086 raft_consensus.cc:374] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:27.623124 21086 raft_consensus.cc:729] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a68856f2b0824b31be4e6f485aa4263b, State: Initialized, Role: FOLLOWER
I20260807 04:09:27.623839 21086 consensus_queue.cc:260] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68856f2b0824b31be4e6f485aa4263b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34981 } }
I20260807 04:09:27.624096 21086 raft_consensus.cc:388] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:27.624261 21086 raft_consensus.cc:482] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:27.624467 21086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:27.627029 21086 raft_consensus.cc:504] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68856f2b0824b31be4e6f485aa4263b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34981 } }
I20260807 04:09:27.627569 21086 leader_election.cc:302] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a68856f2b0824b31be4e6f485aa4263b; no voters:
I20260807 04:09:27.628149 21086 leader_election.cc:288] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:27.628297 21091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:27.629159 21091 raft_consensus.cc:686] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [term 1 LEADER]: Becoming Leader. State: Replica: a68856f2b0824b31be4e6f485aa4263b, State: Running, Role: LEADER
I20260807 04:09:27.629722 21091 consensus_queue.cc:237] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68856f2b0824b31be4e6f485aa4263b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34981 } }
I20260807 04:09:27.630507 21086 sys_catalog.cc:564] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:27.633082 21093 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a68856f2b0824b31be4e6f485aa4263b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68856f2b0824b31be4e6f485aa4263b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34981 } } }
I20260807 04:09:27.633499 21093 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:27.633533 21094 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a68856f2b0824b31be4e6f485aa4263b. Latest consensus state: current_term: 1 leader_uuid: "a68856f2b0824b31be4e6f485aa4263b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68856f2b0824b31be4e6f485aa4263b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34981 } } }
I20260807 04:09:27.633880 21094 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:27.635710 21100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:27.641252 21100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:27.650725 21100 catalog_manager.cc:1347] Generated new cluster ID: 5cf1f779ce564ca59c2f4f6dd05efee8
I20260807 04:09:27.650861 21100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:27.667671 21100 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:27.669358 21100 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:27.675172 21100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Generated new TSK 0
I20260807 04:09:27.676216 21100 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:27.733796 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34981
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:28.081729 21113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:28.082069 21113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:28.082257 21113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:28.089632 21113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:28.089790 21113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:28.089952 21113 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 04:09:28.090046 21113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:09:28.090186 21113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:28.100124 21113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:34981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:28.101541 21113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:28.102934 21113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:28.116916 21118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:28.116856 21119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:28.116794 21121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:28.117776 21113 server_base.cc:1034] running on GCE node
I20260807 04:09:28.118460 21113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:28.119591 21113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:28.120855 21113 hybrid_clock.cc:648] HybridClock initialized: now 1786075768120781 us; error 53 us; skew 500 ppm
I20260807 04:09:28.121237 21113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:28 dist-test-slave-8n49 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075768, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:28.132651 21113 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:28.134557 21113 webserver.cc:460] Webserver started at http://127.18.193.1:36063/ using document root <none> and password file <none>
I20260807 04:09:28.135352 21113 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:28.135517 21113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:28.136061 21113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:28.139837 21113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "0f7a7bc2600743e1955b455be4056545"
format_stamp: "Formatted at 2026-08-07 04:09:28 on dist-test-slave-8n49"
server_key: "4a2b0a1419c83812af01830d640358d7"
server_key_iv: "d818fc018642fbae31bac48cd21c360c"
server_key_version: "encryptionkey@0"
I20260807 04:09:28.140714 21113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "0f7a7bc2600743e1955b455be4056545"
format_stamp: "Formatted at 2026-08-07 04:09:28 on dist-test-slave-8n49"
server_key: "4a2b0a1419c83812af01830d640358d7"
server_key_iv: "d818fc018642fbae31bac48cd21c360c"
server_key_version: "encryptionkey@0"
I20260807 04:09:28.146389 21113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:09:28.150359 21128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:28.151917 21113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:09:28.152089 21113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "0f7a7bc2600743e1955b455be4056545"
format_stamp: "Formatted at 2026-08-07 04:09:28 on dist-test-slave-8n49"
server_key: "4a2b0a1419c83812af01830d640358d7"
server_key_iv: "d818fc018642fbae31bac48cd21c360c"
server_key_version: "encryptionkey@0"
I20260807 04:09:28.152266 21113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:28.174540 21113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:28.179037 21113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:28.179286 21113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:28.182030 21113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:28.182189 21113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:28.182320 21113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:28.182426 21113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:09:28 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075768, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:28.208149 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.182664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:52305 (local address 127.18.193.62:34981)
0807 04:09:28.183340 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:28.183348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:28.184599 (+ 1251us) server_negotiation.cc:407] Connection header received
0807 04:09:28.187214 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:28.187223 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:28.187323 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:28.187420 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:28.190084 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.191393 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.194050 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.194658 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.194837 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.200066 (+ 5229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:28.200073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:28.200076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:28.200111 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:28.203828 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.204887 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.204896 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.204899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.205091 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.205733 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.205737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.205739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.206030 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:28.206144 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:28.207359 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0807 04:09:28.207650 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":440,"threads_started":1}
I20260807 04:09:28.210104 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.183437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34981 (local address 127.18.193.1:52305)
0807 04:09:28.184395 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:28.184444 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:28.186804 (+ 2360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:28.188032 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:28.188046 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:28.188844 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:28.189803 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.189840 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.191635 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.191643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.193850 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.193869 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.195192 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.195196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.195570 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.196432 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:28.196465 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:28.199775 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:28.203992 (+ 4217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.204005 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.204024 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.204743 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.205239 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.205247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.205253 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.205594 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.206166 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:28.206178 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:28.206781 (+ 603us) client_negotiation.cc:769] Sending connection context
0807 04:09:28.207175 (+ 394us) client_negotiation.cc:240] Negotiation successful
0807 04:09:28.207942 (+ 767us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":424,"threads_started":1}
I20260807 04:09:28.228490 21113 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:36461
I20260807 04:09:28.228524 21253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:36461 every 8 connection(s)
I20260807 04:09:28.230268 21113 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:28.233062 21254 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34981
I20260807 04:09:28.233330 21254 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:28.234073 21254 heartbeater.cc:511] Master 127.18.193.62:34981 requested a full tablet report, sending...
I20260807 04:09:28.234483 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.217003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:51503 (local address 127.18.193.62:34981)
0807 04:09:28.217210 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:28.217218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:28.218162 (+ 944us) server_negotiation.cc:407] Connection header received
0807 04:09:28.218447 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:28.218453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:28.218547 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:28.218659 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:28.219957 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.221342 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.224216 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.225075 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.225231 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.227452 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:28.227460 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:28.227467 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:28.227520 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:28.231144 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.232187 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.232193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.232197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.232364 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.232836 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.232841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.232845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.233215 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:28.233369 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:28.233811 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 04:09:28.234048 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 04:09:28.234555 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.217379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34981 (local address 127.18.193.1:51503)
0807 04:09:28.218023 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:28.218049 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:28.218231 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:28.218819 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:28.218823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:28.219286 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:28.219740 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.219758 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.221551 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.221557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.223959 (+ 2402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.223979 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.225343 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.225353 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.225587 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.226379 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:28.226420 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:28.227244 (+ 824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:28.231340 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.231349 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.231355 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.232053 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.232525 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.232529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.232532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.232721 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.233341 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:28.233347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:28.233550 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 04:09:28.233708 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 04:09:28.234081 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":350,"threads_started":1}
I20260807 04:09:28.236748 21038 ts_manager.cc:194] Registered new tserver with Master: 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461)
I20260807 04:09:28.238178 21038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:52305
I20260807 04:09:28.239501 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 21113
I20260807 04:09:28.239735 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:28.240607 19204 external_mini_cluster.cc:1428] Setting key 6001203e33e21238852ba9274e2972fd
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:28.289242 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34981
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:28.629505 21258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:28.629817 21258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:28.629922 21258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:28.637573 21258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:28.637698 21258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:28.637784 21258 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 04:09:28.637841 21258 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:09:28.637905 21258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:28.648710 21258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:34981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:28.650041 21258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:28.651356 21258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:28.665311 21264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:28.665268 21266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:28.665347 21263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:28.666222 21258 server_base.cc:1034] running on GCE node
I20260807 04:09:28.666879 21258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:28.667899 21258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:28.669139 21258 hybrid_clock.cc:648] HybridClock initialized: now 1786075768669095 us; error 59 us; skew 500 ppm
I20260807 04:09:28.669487 21258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:28 dist-test-slave-8n49 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075768, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:28.672834 21258 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:28.674422 21258 webserver.cc:460] Webserver started at http://127.18.193.2:44465/ using document root <none> and password file <none>
I20260807 04:09:28.675166 21258 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:28.675326 21258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:28.675696 21258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:28.679047 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "1b8aa6bbbbea48ba9f45add94f6fe800"
format_stamp: "Formatted at 2026-08-07 04:09:28 on dist-test-slave-8n49"
server_key: "c5d329a68305872c068022e13a6b8d6a"
server_key_iv: "9458444ab4a3ea41995ae9d3952678cf"
server_key_version: "encryptionkey@0"
I20260807 04:09:28.679824 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "1b8aa6bbbbea48ba9f45add94f6fe800"
format_stamp: "Formatted at 2026-08-07 04:09:28 on dist-test-slave-8n49"
server_key: "c5d329a68305872c068022e13a6b8d6a"
server_key_iv: "9458444ab4a3ea41995ae9d3952678cf"
server_key_version: "encryptionkey@0"
I20260807 04:09:28.685534 21258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 04:09:28.689119 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:28.690470 21258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:09:28.690624 21258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "1b8aa6bbbbea48ba9f45add94f6fe800"
format_stamp: "Formatted at 2026-08-07 04:09:28 on dist-test-slave-8n49"
server_key: "c5d329a68305872c068022e13a6b8d6a"
server_key_iv: "9458444ab4a3ea41995ae9d3952678cf"
server_key_version: "encryptionkey@0"
I20260807 04:09:28.690795 21258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:28.711187 21258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:28.716041 21258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:28.716296 21258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:28.718988 21258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:28.719120 21258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:28.719264 21258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:28.719363 21258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:09:28 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075768, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:28.744480 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.719885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:57641 (local address 127.18.193.62:34981)
0807 04:09:28.720114 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:28.720121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:28.721572 (+ 1451us) server_negotiation.cc:407] Connection header received
0807 04:09:28.723604 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:28.723612 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:28.723721 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:28.723870 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:28.726343 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.727677 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.730259 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.730854 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.731006 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.736098 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:28.736107 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:28.736114 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:28.736171 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:28.740301 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.741362 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.741372 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.741378 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.741579 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.742081 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.742089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.742095 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.742570 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:28.742779 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:28.743581 (+ 802us) server_negotiation.cc:299] Negotiation successful
0807 04:09:28.743911 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 04:09:28.745234 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.720410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34981 (local address 127.18.193.2:57641)
0807 04:09:28.721380 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:28.721429 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:28.723313 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:28.724160 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:28.724171 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:28.725129 (+ 958us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:28.726101 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.726133 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.727872 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.727879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.730088 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.730101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.731199 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.731218 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.731452 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.732183 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:28.732205 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:28.735749 (+ 3544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:28.740470 (+ 4721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.740483 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.740497 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.741168 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.741722 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.741726 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.741728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.741951 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.742747 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:28.742753 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:28.743137 (+ 384us) client_negotiation.cc:769] Sending connection context
0807 04:09:28.743384 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 04:09:28.743817 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":457,"threads_started":1}
I20260807 04:09:28.774586 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.755638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34981 (local address 127.18.193.2:48121)
0807 04:09:28.756662 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:28.756700 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:28.756969 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:28.757437 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:28.757442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:28.758088 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:28.758571 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.758584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.760461 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.760470 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.762919 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:28.762952 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.764168 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.764174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:28.764307 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.765271 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:28.765304 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:28.766529 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:28.770847 (+ 4318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.770853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.770857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.771521 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.772190 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:28.772201 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:28.772205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:28.772358 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:28.773273 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:28.773280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:28.773520 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 04:09:28.773759 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 04:09:28.774165 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":408,"threads_started":1}
I20260807 04:09:28.774751 21258 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:43837
I20260807 04:09:28.774803 21397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:43837 every 8 connection(s)
I20260807 04:09:28.775489 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:28.755621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:48121 (local address 127.18.193.62:34981)
0807 04:09:28.755932 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:28.755942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:28.756808 (+ 866us) server_negotiation.cc:407] Connection header received
0807 04:09:28.757163 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:28.757169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:28.757260 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:28.757391 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:28.758840 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.760192 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.763195 (+ 3003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:28.763903 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:28.764073 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:28.766917 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:28.766926 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:28.766932 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:28.766979 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:28.770674 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.771799 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.771806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.771810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.772029 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:28.772511 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:28.772516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:28.772520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:28.773095 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:28.773267 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:28.774980 (+ 1713us) server_negotiation.cc:299] Negotiation successful
0807 04:09:28.775260 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:09:28.776209 21258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:28.779282 21398 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34981
I20260807 04:09:28.779580 21398 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:28.780277 21398 heartbeater.cc:511] Master 127.18.193.62:34981 requested a full tablet report, sending...
I20260807 04:09:28.780970 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 21258
I20260807 04:09:28.781211 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:28.781700 19204 external_mini_cluster.cc:1428] Setting key eff9038ca92fad062caa08cb1041a740
I20260807 04:09:28.781841 21038 ts_manager.cc:194] Registered new tserver with Master: 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837)
I20260807 04:09:28.782887 21038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:57641
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:28.832420 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34981
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:29.171729 21402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:29.172060 21402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:29.172240 21402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:29.179705 21402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:29.179863 21402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:29.180027 21402 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 04:09:29.180119 21402 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:09:29.180255 21402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:29.190182 21402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:34981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:29.191527 21402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:29.192986 21402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:29.206233 21407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:29.206210 21410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:29.206489 21408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:29.206938 21402 server_base.cc:1034] running on GCE node
I20260807 04:09:29.207640 21402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:29.208611 21402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:29.209861 21402 hybrid_clock.cc:648] HybridClock initialized: now 1786075769209823 us; error 32 us; skew 500 ppm
I20260807 04:09:29.210215 21402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:29 dist-test-slave-8n49 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075769, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:29.213426 21402 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:29.214951 21402 webserver.cc:460] Webserver started at http://127.18.193.3:41359/ using document root <none> and password file <none>
I20260807 04:09:29.215673 21402 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:29.215812 21402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:29.216144 21402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:29.219170 21402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "e87708af134a40cfbd9c0d62cd57bed5"
format_stamp: "Formatted at 2026-08-07 04:09:29 on dist-test-slave-8n49"
server_key: "3ee9613b6f0648fa386410437293a9f7"
server_key_iv: "f753d2927206a2381b8f6ef678924aab"
server_key_version: "encryptionkey@0"
I20260807 04:09:29.219944 21402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "e87708af134a40cfbd9c0d62cd57bed5"
format_stamp: "Formatted at 2026-08-07 04:09:29 on dist-test-slave-8n49"
server_key: "3ee9613b6f0648fa386410437293a9f7"
server_key_iv: "f753d2927206a2381b8f6ef678924aab"
server_key_version: "encryptionkey@0"
I20260807 04:09:29.225097 21402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:09:29.228740 21417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:29.230189 21402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:09:29.230365 21402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "e87708af134a40cfbd9c0d62cd57bed5"
format_stamp: "Formatted at 2026-08-07 04:09:29 on dist-test-slave-8n49"
server_key: "3ee9613b6f0648fa386410437293a9f7"
server_key_iv: "f753d2927206a2381b8f6ef678924aab"
server_key_version: "encryptionkey@0"
I20260807 04:09:29.230569 21402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:29.241611 21254 heartbeater.cc:503] Master 127.18.193.62:34981 was elected leader, sending a full tablet report...
I20260807 04:09:29.254333 21402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:29.259251 21402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:29.259582 21402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:29.262418 21402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:29.262555 21402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:29.262699 21402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:29.262794 21402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:09:29 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075769, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:29.289680 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.263297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:39323 (local address 127.18.193.62:34981)
0807 04:09:29.263508 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.263514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.265101 (+ 1587us) server_negotiation.cc:407] Connection header received
0807 04:09:29.267713 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.267722 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.267828 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.267914 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:29.270709 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.271825 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.274480 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.275095 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.275260 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.280919 (+ 5659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:29.280925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:29.280928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:29.280967 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:29.284892 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.286035 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.286042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.286046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.286256 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.286998 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.287006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.287013 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.287403 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:29.287589 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.288869 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.289189 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:09:29.291661 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.263900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34981 (local address 127.18.193.3:39323)
0807 04:09:29.264898 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.264956 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.267288 (+ 2332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.268528 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.268588 (+ 60us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.269414 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:29.270501 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.270525 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.272113 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.272119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.274305 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.274318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.275396 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.275404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.275677 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.276646 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:29.276701 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:29.280544 (+ 3843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:29.285069 (+ 4525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:29.285085 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:29.285104 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:29.285884 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:29.286413 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:29.286422 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:29.286428 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:29.286820 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:29.287557 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:29.287566 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:29.288248 (+ 682us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.288689 (+ 441us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.289524 (+ 835us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":429,"threads_started":1}
I20260807 04:09:29.306375 21402 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:42245
I20260807 04:09:29.306478 21541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:42245 every 8 connection(s)
I20260807 04:09:29.308282 21402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:29.310879 21542 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34981
I20260807 04:09:29.311118 21542 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:29.311976 21542 heartbeater.cc:511] Master 127.18.193.62:34981 requested a full tablet report, sending...
I20260807 04:09:29.313581 21039 ts_manager.cc:194] Registered new tserver with Master: e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245)
I20260807 04:09:29.314751 21039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:39323
I20260807 04:09:29.315316 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.299654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:40643 (local address 127.18.193.62:34981)
0807 04:09:29.299851 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.299857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.300743 (+ 886us) server_negotiation.cc:407] Connection header received
0807 04:09:29.301020 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.301026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.301159 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.301276 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:29.303021 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.303745 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.306040 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.306759 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.306860 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.309340 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:29.309347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:29.309351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:29.309390 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:29.312424 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.313299 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.313306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.313309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.313446 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.313862 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.313867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.313873 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.314189 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:29.314310 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.314741 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.314965 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:09:29.315441 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.299835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34981 (local address 127.18.193.3:40643)
0807 04:09:29.300616 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.300640 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.300873 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.301342 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.301350 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.302158 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:29.302837 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.302854 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.303904 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.303914 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.305845 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.305867 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.306981 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.306988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.307178 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.307857 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:29.307878 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:29.309069 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:29.312583 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:29.312591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:29.312598 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:29.313168 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:29.313570 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:29.313576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:29.313579 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:29.313735 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:29.314315 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:29.314322 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:29.314501 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.314647 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.314958 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":379,"threads_started":1}
I20260807 04:09:29.315963 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 21402
I20260807 04:09:29.316145 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:29.316752 19204 external_mini_cluster.cc:1428] Setting key 14c34b11452c62d0124e3a6958b983dd
I20260807 04:09:29.319967 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:09:29 dist-test-slave-8n49 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:09:29 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:29.375572 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.358747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50150 (local address 127.18.193.62:34981)
0807 04:09:29.359070 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.359078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.359286 (+ 208us) server_negotiation.cc:407] Connection header received
0807 04:09:29.359505 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.359511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.359585 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.359693 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:29.361227 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.361962 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.363226 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.363644 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.363787 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.367755 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:29.367760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:29.367764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:29.367801 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:29.372052 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.373048 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.373056 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.373064 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.373249 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.373720 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.373725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.373728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.374151 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:29.374430 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.374802 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.375095 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 04:09:29.380120 21039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:29.387933 21039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:29.413244 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.401335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:42245 (local address 127.0.0.1:51420)
0807 04:09:29.403616 (+ 2281us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.403643 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.403780 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.405217 (+ 1437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.405226 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.405258 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:29.405717 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.405738 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.407864 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.407873 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.410480 (+ 2607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.410493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.411950 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.411957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.412266 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.412335 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.412506 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.412775 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2100,"mutex_wait_us":3,"thread_start_us":1414,"threads_started":1}
I20260807 04:09:29.413849 21558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.402084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.18.193.3:42245)
0807 04:09:29.404590 (+ 2506us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.404599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.404638 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:09:29.404737 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.404742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.404994 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.405203 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:29.406187 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.407541 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.410661 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.411698 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.412716 (+ 1018us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.412992 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.413148 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.413334 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2300,"thread_start_us":2123,"threads_started":1}
I20260807 04:09:29.414420 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.400865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:43837 (local address 127.0.0.1:38324)
0807 04:09:29.402136 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.402178 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.402384 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.405573 (+ 3189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.405582 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.405630 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:29.406265 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.406283 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.408811 (+ 2528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.408817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.411493 (+ 2676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.411512 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.413596 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.413603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.413769 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.413814 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.413886 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.414026 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"thread_start_us":825,"threads_started":1}
I20260807 04:09:29.415370 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.401076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38324 (local address 127.18.193.2:43837)
0807 04:09:29.404735 (+ 3659us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.404745 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.404779 (+ 34us) server_negotiation.cc:407] Connection header received
0807 04:09:29.404911 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.404923 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.405266 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.406530 (+ 1264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:29.406817 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.408664 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.411716 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.413326 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.414763 (+ 1437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.414916 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.414972 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.415125 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3376,"spinlock_wait_cycles":23808,"thread_start_us":3348,"threads_started":1}
I20260807 04:09:29.417773 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.403304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.18.193.1:36461)
0807 04:09:29.408328 (+ 5024us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.408337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.408368 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:09:29.408499 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.408509 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.408800 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.409008 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:29.409902 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.411584 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.414121 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.416731 (+ 2610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.416884 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.417094 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.417321 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.417482 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4851,"thread_start_us":4701,"threads_started":1}
I20260807 04:09:29.417788 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.402642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36461 (local address 127.0.0.1:35116)
0807 04:09:29.403827 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.403852 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.404021 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.409031 (+ 5010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.409036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.409072 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:29.409653 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.409667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.411788 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.411795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.413862 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.413880 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.416949 (+ 3069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.416956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.417137 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.417168 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.417276 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.417413 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":1,"thread_start_us":1220,"threads_started":1}
I20260807 04:09:29.418637 21469 tablet_service.cc:1467] Processing CreateTablet for tablet 609ddd6cabe04f18a958dc45f31abf31 (DEFAULT_TABLE table=test-table [id=3e648729bd704d6a800a38647af22914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:29.418637 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 609ddd6cabe04f18a958dc45f31abf31 (DEFAULT_TABLE table=test-table [id=3e648729bd704d6a800a38647af22914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:29.420133 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609ddd6cabe04f18a958dc45f31abf31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:29.420133 21469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609ddd6cabe04f18a958dc45f31abf31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:29.422318 21181 tablet_service.cc:1467] Processing CreateTablet for tablet 609ddd6cabe04f18a958dc45f31abf31 (DEFAULT_TABLE table=test-table [id=3e648729bd704d6a800a38647af22914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:29.423457 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609ddd6cabe04f18a958dc45f31abf31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:29.436092 21562 tablet_bootstrap.cc:492] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Bootstrap starting.
I20260807 04:09:29.438855 21561 tablet_bootstrap.cc:492] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Bootstrap starting.
I20260807 04:09:29.440447 21562 tablet_bootstrap.cc:654] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:29.441749 21562 log.cc:824] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:29.442764 21563 tablet_bootstrap.cc:492] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Bootstrap starting.
I20260807 04:09:29.443427 21561 tablet_bootstrap.cc:654] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:29.444752 21561 log.cc:824] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:29.446715 21563 tablet_bootstrap.cc:654] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:29.446851 21562 tablet_bootstrap.cc:492] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: No bootstrap required, opened a new log
I20260807 04:09:29.447158 21562 ts_tablet_manager.cc:1397] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 04:09:29.447702 21561 tablet_bootstrap.cc:492] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: No bootstrap required, opened a new log
I20260807 04:09:29.448026 21561 ts_tablet_manager.cc:1397] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 04:09:29.448093 21563 log.cc:824] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:29.451701 21563 tablet_bootstrap.cc:492] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: No bootstrap required, opened a new log
I20260807 04:09:29.451989 21563 ts_tablet_manager.cc:1397] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260807 04:09:29.457816 21561 raft_consensus.cc:348] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.458271 21561 raft_consensus.cc:374] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:29.458391 21561 raft_consensus.cc:729] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7a7bc2600743e1955b455be4056545, State: Initialized, Role: FOLLOWER
I20260807 04:09:29.458459 21562 raft_consensus.cc:348] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.459118 21562 raft_consensus.cc:374] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:29.459233 21562 raft_consensus.cc:729] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8aa6bbbbea48ba9f45add94f6fe800, State: Initialized, Role: FOLLOWER
I20260807 04:09:29.459162 21561 consensus_queue.cc:260] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.459854 21562 consensus_queue.cc:260] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.462725 21561 ts_tablet_manager.cc:1428] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260807 04:09:29.464756 21563 raft_consensus.cc:348] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.465293 21563 raft_consensus.cc:374] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:29.465404 21563 raft_consensus.cc:729] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87708af134a40cfbd9c0d62cd57bed5, State: Initialized, Role: FOLLOWER
I20260807 04:09:29.465540 21398 heartbeater.cc:503] Master 127.18.193.62:34981 was elected leader, sending a full tablet report...
I20260807 04:09:29.466238 21562 ts_tablet_manager.cc:1428] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent starting tablet: real 0.019s user 0.008s sys 0.012s
I20260807 04:09:29.466094 21563 consensus_queue.cc:260] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.467407 21542 heartbeater.cc:503] Master 127.18.193.62:34981 was elected leader, sending a full tablet report...
I20260807 04:09:29.468048 21563 ts_tablet_manager.cc:1428] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Time spent starting tablet: real 0.016s user 0.015s sys 0.002s
W20260807 04:09:29.483942 21255 tablet.cc:2369] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:09:29.528587 21399 tablet.cc:2369] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:09:29.558620 21567 raft_consensus.cc:482] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:29.558977 21567 raft_consensus.cc:504] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
W20260807 04:09:29.560153 21543 tablet.cc:2369] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:09:29.561167 21567 leader_election.cc:288] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837), e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245)
I20260807 04:09:29.568151 21558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.561703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:38975 (local address 127.18.193.3:42245)
0807 04:09:29.561994 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.562000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.562526 (+ 526us) server_negotiation.cc:407] Connection header received
0807 04:09:29.562750 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.562755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.562815 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.562921 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:29.563547 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.564269 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.566294 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.567077 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.567280 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.567364 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.567728 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.567869 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 04:09:29.568162 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.561522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:34419 (local address 127.18.193.2:43837)
0807 04:09:29.561777 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.561787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.562558 (+ 771us) server_negotiation.cc:407] Connection header received
0807 04:09:29.562780 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.562785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.562844 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.562944 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:29.563617 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.564358 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.566283 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.567090 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.567287 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.567375 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.567724 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.567869 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:09:29.568436 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.561453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:43837 (local address 127.18.193.1:34419)
0807 04:09:29.562427 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.562442 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.562622 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.562957 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.562961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.562985 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:29.563440 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.563449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.564521 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.564530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.566110 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.566124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.567383 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.567392 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.567533 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.567579 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.567687 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.567873 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":50,"spinlock_wait_cycles":2816,"thread_start_us":495,"threads_started":1}
I20260807 04:09:29.568910 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.561415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:42245 (local address 127.18.193.1:38975)
0807 04:09:29.562392 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.562422 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.562589 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.562936 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.562941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.562965 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:29.563381 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.563394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.564404 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.564412 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.566129 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.566142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.567334 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.567341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.567519 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.567591 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.567699 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.567881 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":103,"thread_start_us":760,"threads_started":1}
I20260807 04:09:29.569581 21352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609ddd6cabe04f18a958dc45f31abf31" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" is_pre_election: true
I20260807 04:09:29.569775 21490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609ddd6cabe04f18a958dc45f31abf31" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87708af134a40cfbd9c0d62cd57bed5" is_pre_election: true
I20260807 04:09:29.570158 21352 raft_consensus.cc:2454] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 0.
I20260807 04:09:29.570302 21490 raft_consensus.cc:2454] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 0.
I20260807 04:09:29.570878 21129 leader_election.cc:302] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, 1b8aa6bbbbea48ba9f45add94f6fe800, e87708af134a40cfbd9c0d62cd57bed5; no voters:
I20260807 04:09:29.571328 21567 raft_consensus.cc:2781] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:29.571460 21567 raft_consensus.cc:482] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:29.571542 21567 raft_consensus.cc:3037] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:29.573840 21567 raft_consensus.cc:504] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.574656 21567 leader_election.cc:288] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 election: Requested vote from peers 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837), e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245)
I20260807 04:09:29.575145 21352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609ddd6cabe04f18a958dc45f31abf31" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800"
I20260807 04:09:29.575150 21490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609ddd6cabe04f18a958dc45f31abf31" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87708af134a40cfbd9c0d62cd57bed5"
I20260807 04:09:29.575379 21490 raft_consensus.cc:3037] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:29.575380 21352 raft_consensus.cc:3037] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:29.578712 21352 raft_consensus.cc:2454] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 1.
I20260807 04:09:29.578712 21490 raft_consensus.cc:2454] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 1.
I20260807 04:09:29.579170 21130 leader_election.cc:302] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, e87708af134a40cfbd9c0d62cd57bed5; no voters:
I20260807 04:09:29.579531 21567 raft_consensus.cc:2781] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:29.580018 21567 raft_consensus.cc:686] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 1 LEADER]: Becoming Leader. State: Replica: 0f7a7bc2600743e1955b455be4056545, State: Running, Role: LEADER
I20260807 04:09:29.580536 21567 consensus_queue.cc:237] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } }
I20260807 04:09:29.585727 21039 catalog_manager.cc:5495] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f7a7bc2600743e1955b455be4056545 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } health_report { overall_health: HEALTHY } } }
Aug 07 04:09:29 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075767, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:29.651603 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.630640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.18.193.62:34981)
0807 04:09:29.630876 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:29.630885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:29.630908 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:09:29.630975 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:29.630985 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:29.631049 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:29.631146 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:29.633722 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.635170 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.638976 (+ 3806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.639421 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.639524 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.644722 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:29.644728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:29.644731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:29.644766 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:29.648530 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.649421 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.649426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.649430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.649566 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:29.649929 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:29.649934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:29.649937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:29.650239 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:29.650422 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:29.650894 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 04:09:29.651282 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":77}
I20260807 04:09:29.651644 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:29.628104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34981 (local address 127.0.0.1:50160)
0807 04:09:29.629173 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:29.629191 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:29.629319 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:29.631193 (+ 1874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:29.631197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:29.633119 (+ 1922us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:29.633528 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.633547 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.635407 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.635415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.637202 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:29.637217 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:29.639677 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:29.639691 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:29.639946 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:29.640944 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:29.640981 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:29.644444 (+ 3463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:29.648738 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:29.648760 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:29.648767 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:29.649288 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:29.649675 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:29.649681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:29.649684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:29.649814 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:29.650383 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:29.650391 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:29.650571 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 04:09:29.650783 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 04:09:29.651137 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"thread_start_us":569,"threads_started":1}
I20260807 04:09:29.657492 21039 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:50160:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 04:09:29.658555 21039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:29.668100 21325 tablet_service.cc:1467] Processing CreateTablet for tablet c37ccf8e083f4d548d88823808968644 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5372dffdc89c49fb9ab9f886174357e1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:09:29.667878 21181 tablet_service.cc:1467] Processing CreateTablet for tablet c37ccf8e083f4d548d88823808968644 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5372dffdc89c49fb9ab9f886174357e1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:09:29.668395 21469 tablet_service.cc:1467] Processing CreateTablet for tablet c37ccf8e083f4d548d88823808968644 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5372dffdc89c49fb9ab9f886174357e1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:09:29.668984 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c37ccf8e083f4d548d88823808968644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:29.669008 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c37ccf8e083f4d548d88823808968644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:29.669276 21469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c37ccf8e083f4d548d88823808968644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:29.674393 21562 tablet_bootstrap.cc:492] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: Bootstrap starting.
I20260807 04:09:29.676872 21563 tablet_bootstrap.cc:492] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: Bootstrap starting.
I20260807 04:09:29.677073 21562 tablet_bootstrap.cc:654] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:29.678274 21563 tablet_bootstrap.cc:654] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:29.679705 21562 tablet_bootstrap.cc:492] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: No bootstrap required, opened a new log
I20260807 04:09:29.679991 21562 ts_tablet_manager.cc:1397] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 04:09:29.680348 21563 tablet_bootstrap.cc:492] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: No bootstrap required, opened a new log
I20260807 04:09:29.680511 21563 ts_tablet_manager.cc:1397] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 04:09:29.681133 21563 raft_consensus.cc:348] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:29.681100 21562 raft_consensus.cc:348] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:29.681356 21563 raft_consensus.cc:374] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:29.681444 21562 raft_consensus.cc:374] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:29.681492 21563 raft_consensus.cc:729] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87708af134a40cfbd9c0d62cd57bed5, State: Initialized, Role: FOLLOWER
I20260807 04:09:29.681609 21562 raft_consensus.cc:729] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8aa6bbbbea48ba9f45add94f6fe800, State: Initialized, Role: FOLLOWER
I20260807 04:09:29.681754 21563 consensus_queue.cc:260] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:29.681895 21562 consensus_queue.cc:260] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:29.682240 21569 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:29.682578 21569 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:29.682464 21568 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:29.682894 21568 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:29.683523 21562 ts_tablet_manager.cc:1428] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 04:09:29.684193 21561 tablet_bootstrap.cc:492] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: Bootstrap starting.
I20260807 04:09:29.684857 21563 ts_tablet_manager.cc:1428] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260807 04:09:29.685943 21561 tablet_bootstrap.cc:654] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:29.688812 21561 tablet_bootstrap.cc:492] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: No bootstrap required, opened a new log
I20260807 04:09:29.688971 21561 ts_tablet_manager.cc:1397] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 04:09:29.689841 21561 raft_consensus.cc:348] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:29.690101 21561 raft_consensus.cc:374] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:29.690197 21561 raft_consensus.cc:729] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7a7bc2600743e1955b455be4056545, State: Initialized, Role: FOLLOWER
I20260807 04:09:29.690421 21561 consensus_queue.cc:260] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:29.690824 21567 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:29.690994 21561 ts_tablet_manager.cc:1428] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 04:09:29.691143 21567 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:30.006641 21567 raft_consensus.cc:482] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:30.006904 21567 raft_consensus.cc:504] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.007608 21567 leader_election.cc:288] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837), e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245)
I20260807 04:09:30.008276 21490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c37ccf8e083f4d548d88823808968644" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87708af134a40cfbd9c0d62cd57bed5" is_pre_election: true
I20260807 04:09:30.008347 21352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c37ccf8e083f4d548d88823808968644" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" is_pre_election: true
I20260807 04:09:30.008592 21490 raft_consensus.cc:2454] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 0.
I20260807 04:09:30.008738 21352 raft_consensus.cc:2454] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 0.
I20260807 04:09:30.009132 21130 leader_election.cc:302] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, e87708af134a40cfbd9c0d62cd57bed5; no voters:
I20260807 04:09:30.009465 21567 raft_consensus.cc:2781] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:30.009603 21567 raft_consensus.cc:482] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:30.009732 21567 raft_consensus.cc:3037] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:30.013818 21567 raft_consensus.cc:504] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.014745 21567 leader_election.cc:288] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 election: Requested vote from peers 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837), e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245)
I20260807 04:09:30.015265 21352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c37ccf8e083f4d548d88823808968644" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800"
I20260807 04:09:30.015276 21490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c37ccf8e083f4d548d88823808968644" candidate_uuid: "0f7a7bc2600743e1955b455be4056545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87708af134a40cfbd9c0d62cd57bed5"
I20260807 04:09:30.015539 21352 raft_consensus.cc:3037] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:30.015576 21490 raft_consensus.cc:3037] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:30.017696 21352 raft_consensus.cc:2454] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 1.
I20260807 04:09:30.018180 21129 leader_election.cc:302] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, 1b8aa6bbbbea48ba9f45add94f6fe800; no voters:
I20260807 04:09:30.018491 21567 raft_consensus.cc:2781] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:30.018695 21567 raft_consensus.cc:686] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [term 1 LEADER]: Becoming Leader. State: Replica: 0f7a7bc2600743e1955b455be4056545, State: Running, Role: LEADER
I20260807 04:09:30.018959 21567 consensus_queue.cc:237] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.019591 21490 raft_consensus.cc:2454] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f7a7bc2600743e1955b455be4056545 in term 1.
I20260807 04:09:30.021576 21572 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: TxnStatusTablet state changed. Reason: New leader 0f7a7bc2600743e1955b455be4056545. Latest consensus state: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:30.021992 21572 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:09:30.022758 21585 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 04:09:30.023138 21585 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 04:09:30.023495 21039 catalog_manager.cc:5495] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f7a7bc2600743e1955b455be4056545 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260807 04:09:30.024540 21572 consensus_queue.cc:1035] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:30.049230 21567 consensus_queue.cc:1035] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:30.081033 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.074045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.18.193.1:36461)
0807 04:09:30.074596 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.074601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.074618 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:09:30.074705 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.074708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.074779 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.074909 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:30.075398 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.076506 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.078492 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.079019 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.079139 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.079717 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.080332 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.080512 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.080724 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.080796 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":299,"threads_started":1}
I20260807 04:09:30.081132 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.073961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36461 (local address 127.0.0.1:35126)
0807 04:09:30.074270 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:30.074297 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:30.074465 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:30.074972 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:30.074978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:30.074992 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:30.075201 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.075213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.076681 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.076688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.078308 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.078325 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.079208 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.079215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.079423 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.079507 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.080498 (+ 991us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.080525 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 04:09:30.080660 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 04:09:30.080825 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 04:09:30.088150 21352 raft_consensus.cc:1261] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Refusing update from remote peer 0f7a7bc2600743e1955b455be4056545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:30.088147 21490 raft_consensus.cc:1261] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Refusing update from remote peer 0f7a7bc2600743e1955b455be4056545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:30.088881 21567 consensus_queue.cc:1035] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:30.089198 21572 consensus_queue.cc:1035] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:30.096159 21568 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: TxnStatusTablet state changed. Reason: New leader 0f7a7bc2600743e1955b455be4056545. Latest consensus state: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:30.096481 21568 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:30.096418 21569 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: TxnStatusTablet state changed. Reason: New leader 0f7a7bc2600743e1955b455be4056545. Latest consensus state: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:30.096746 21569 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:30.099517 21588 mvcc.cc:205] Tried to move back new op lower bound from 7315766354272833536 to 7315766354000424960. Current Snapshot: MvccSnapshot[applied={T|T < 7315766354272833536}]
I20260807 04:09:30.100461 21567 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:30.100780 21567 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:09:30.101889 21572 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:30.102181 21572 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P 0f7a7bc2600743e1955b455be4056545: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:09:30.108752 21568 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:30.109128 21568 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P 1b8aa6bbbbea48ba9f45add94f6fe800: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:30.115027 21569 tablet_replica.cc:439] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f7a7bc2600743e1955b455be4056545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } } }
I20260807 04:09:30.115317 21569 tablet_replica.cc:442] T c37ccf8e083f4d548d88823808968644 P e87708af134a40cfbd9c0d62cd57bed5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:09:30.124469 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.118465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.18.193.1:36461)
0807 04:09:30.118643 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.118667 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.119197 (+ 530us) server_negotiation.cc:407] Connection header received
0807 04:09:30.119433 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.119447 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.119518 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.119633 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:30.120125 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.121040 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.122588 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.123067 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.123220 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.123699 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.123811 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.123918 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.124097 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.124185 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:09:30.143746 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.137017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:49689 (local address 127.18.193.1:36461)
0807 04:09:30.137213 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.137217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.138002 (+ 785us) server_negotiation.cc:407] Connection header received
0807 04:09:30.138253 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.138259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.138361 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.138476 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:30.139052 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.139936 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.141639 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.142131 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.142287 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.142815 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.142925 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.143031 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.143197 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.143274 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:09:30.143780 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.137079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36461 (local address 127.18.193.1:49689)
0807 04:09:30.137870 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:30.137888 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:30.138101 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:30.138495 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:30.138500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:30.138511 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:30.138880 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.138893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.140079 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.140087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.141460 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.141475 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.142339 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.142348 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.142509 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.142575 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.143031 (+ 456us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.143053 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 04:09:30.143174 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 04:09:30.143317 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":441,"threads_started":1}
I20260807 04:09:30.232292 21038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:30.232645 21038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:30.236804 21038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Sending DeleteTablet for 3 replicas of tablet 609ddd6cabe04f18a958dc45f31abf31
I20260807 04:09:30.238062 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 609ddd6cabe04f18a958dc45f31abf31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:30 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38324
I20260807 04:09:30.238291 21469 tablet_service.cc:1514] Processing DeleteTablet for tablet 609ddd6cabe04f18a958dc45f31abf31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:30 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:51420
I20260807 04:09:30.238709 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet 609ddd6cabe04f18a958dc45f31abf31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:30 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:35116
I20260807 04:09:30.241798 21038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:30.242743 21038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:30.251428 21468 tablet_service.cc:1467] Processing CreateTablet for tablet 424b841def6a40688fef758d0edf9ceb (DEFAULT_TABLE table=test-table [id=9cfac104685f47578f8a681042895118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:30.251760 21324 tablet_service.cc:1467] Processing CreateTablet for tablet 424b841def6a40688fef758d0edf9ceb (DEFAULT_TABLE table=test-table [id=9cfac104685f47578f8a681042895118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:30.251737 21180 tablet_service.cc:1467] Processing CreateTablet for tablet 424b841def6a40688fef758d0edf9ceb (DEFAULT_TABLE table=test-table [id=9cfac104685f47578f8a681042895118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:30.251968 21468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 424b841def6a40688fef758d0edf9ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:30.252310 21324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 424b841def6a40688fef758d0edf9ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:30.252446 21180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 424b841def6a40688fef758d0edf9ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:30.266590 21612 tablet_bootstrap.cc:492] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: Bootstrap starting.
I20260807 04:09:30.267519 21613 tablet_bootstrap.cc:492] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: Bootstrap starting.
I20260807 04:09:30.269580 21610 tablet_replica.cc:330] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: stopping tablet replica
I20260807 04:09:30.270035 21610 raft_consensus.cc:2229] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:09:30.270816 21610 raft_consensus.cc:2258] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:30.271528 21611 tablet_bootstrap.cc:492] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: Bootstrap starting.
I20260807 04:09:30.273372 21608 tablet_replica.cc:330] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: stopping tablet replica
I20260807 04:09:30.273365 21609 tablet_replica.cc:330] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: stopping tablet replica
I20260807 04:09:30.273799 21609 raft_consensus.cc:2229] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:30.273801 21608 raft_consensus.cc:2229] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:30.274170 21609 raft_consensus.cc:2258] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:30.274171 21608 raft_consensus.cc:2258] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:30.274525 21613 tablet_bootstrap.cc:654] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:30.274674 21612 tablet_bootstrap.cc:654] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:30.276376 21608 ts_tablet_manager.cc:1905] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:30.276686 21611 tablet_bootstrap.cc:654] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:30.279292 21610 ts_tablet_manager.cc:1905] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:30.283845 21608 ts_tablet_manager.cc:1918] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:30.284030 21608 log.cc:1196] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/609ddd6cabe04f18a958dc45f31abf31
I20260807 04:09:30.286232 21609 ts_tablet_manager.cc:1905] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:30.292707 21611 tablet_bootstrap.cc:492] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: No bootstrap required, opened a new log
I20260807 04:09:30.292887 21611 ts_tablet_manager.cc:1397] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.002s
I20260807 04:09:30.292906 21608 ts_tablet_manager.cc:1939] T 609ddd6cabe04f18a958dc45f31abf31 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting consensus metadata
I20260807 04:09:30.294885 21611 raft_consensus.cc:348] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.296458 21612 tablet_bootstrap.cc:492] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: No bootstrap required, opened a new log
I20260807 04:09:30.296643 21612 ts_tablet_manager.cc:1397] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.005s
I20260807 04:09:30.297364 21612 raft_consensus.cc:348] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.297621 21612 raft_consensus.cc:374] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:30.297716 21612 raft_consensus.cc:729] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8aa6bbbbea48ba9f45add94f6fe800, State: Initialized, Role: FOLLOWER
I20260807 04:09:30.297955 21612 consensus_queue.cc:260] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.301016 21612 ts_tablet_manager.cc:1428] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 04:09:30.303045 21021 catalog_manager.cc:4841] TS 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837): tablet 609ddd6cabe04f18a958dc45f31abf31 (table test-table [id=3e648729bd704d6a800a38647af22914]) successfully deleted
I20260807 04:09:30.303238 21611 raft_consensus.cc:374] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:30.303391 21611 raft_consensus.cc:729] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87708af134a40cfbd9c0d62cd57bed5, State: Initialized, Role: FOLLOWER
I20260807 04:09:30.303722 21611 consensus_queue.cc:260] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.307592 21613 tablet_bootstrap.cc:492] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: No bootstrap required, opened a new log
I20260807 04:09:30.307827 21613 ts_tablet_manager.cc:1397] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.024s
I20260807 04:09:30.308993 21613 raft_consensus.cc:348] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.309353 21613 raft_consensus.cc:374] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:30.309502 21613 raft_consensus.cc:729] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7a7bc2600743e1955b455be4056545, State: Initialized, Role: FOLLOWER
I20260807 04:09:30.309852 21613 consensus_queue.cc:260] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.310186 21610 ts_tablet_manager.cc:1918] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:30.310384 21610 log.cc:1196] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/609ddd6cabe04f18a958dc45f31abf31
I20260807 04:09:30.310974 21610 ts_tablet_manager.cc:1939] T 609ddd6cabe04f18a958dc45f31abf31 P 0f7a7bc2600743e1955b455be4056545: Deleting consensus metadata
I20260807 04:09:30.311524 21613 ts_tablet_manager.cc:1428] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 04:09:30.312777 21021 catalog_manager.cc:4841] TS 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461): tablet 609ddd6cabe04f18a958dc45f31abf31 (table test-table [id=3e648729bd704d6a800a38647af22914]) successfully deleted
I20260807 04:09:30.314715 21609 ts_tablet_manager.cc:1918] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:30.314879 21609 log.cc:1196] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/609ddd6cabe04f18a958dc45f31abf31
I20260807 04:09:30.315394 21609 ts_tablet_manager.cc:1939] T 609ddd6cabe04f18a958dc45f31abf31 P e87708af134a40cfbd9c0d62cd57bed5: Deleting consensus metadata
I20260807 04:09:30.317164 21022 catalog_manager.cc:4841] TS e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245): tablet 609ddd6cabe04f18a958dc45f31abf31 (table test-table [id=3e648729bd704d6a800a38647af22914]) successfully deleted
I20260807 04:09:30.317521 21611 ts_tablet_manager.cc:1428] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260807 04:09:30.482570 21569 raft_consensus.cc:482] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:30.482849 21569 raft_consensus.cc:504] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.484669 21569 leader_election.cc:288] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461), 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837)
I20260807 04:09:30.497269 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.485069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:53917 (local address 127.18.193.1:36461)
0807 04:09:30.485376 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.485387 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.486012 (+ 625us) server_negotiation.cc:407] Connection header received
0807 04:09:30.486287 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.486297 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.486414 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.486503 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:30.487622 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.489312 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.493191 (+ 3879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.495267 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.495455 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.495586 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.496537 (+ 951us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.496783 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:09:30.498714 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.485059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36461 (local address 127.18.193.3:53917)
0807 04:09:30.485869 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:30.485889 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:30.486066 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:30.486792 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:30.486801 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:30.486841 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:30.487347 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.487364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.489592 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.489601 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.492905 (+ 3304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.492942 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.495946 (+ 3004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.495956 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.496252 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.496312 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 04:09:30.497927 (+ 1615us) client_negotiation.cc:240] Negotiation successful
0807 04:09:30.498165 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":516,"threads_started":1}
I20260807 04:09:30.500792 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424b841def6a40688fef758d0edf9ceb" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7a7bc2600743e1955b455be4056545" is_pre_election: true
I20260807 04:09:30.501331 21204 raft_consensus.cc:2454] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 0.
I20260807 04:09:30.502101 21418 leader_election.cc:302] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, e87708af134a40cfbd9c0d62cd57bed5; no voters:
I20260807 04:09:30.502617 21569 raft_consensus.cc:2781] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:30.502775 21569 raft_consensus.cc:482] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:30.502892 21569 raft_consensus.cc:3037] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:30.506628 21569 raft_consensus.cc:504] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.507375 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.486089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:43837 (local address 127.18.193.3:45141)
0807 04:09:30.499289 (+ 13200us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:30.499317 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:30.499467 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:30.500459 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:30.500463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:30.500497 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:30.500732 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.500744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.502297 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.502354 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.504750 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.504777 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.506557 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.506563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.506776 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.506823 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 04:09:30.506910 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 04:09:30.507068 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13042,"thread_start_us":13676,"threads_started":1}
I20260807 04:09:30.507468 21569 leader_election.cc:288] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 election: Requested vote from peers 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461), 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837)
I20260807 04:09:30.508002 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424b841def6a40688fef758d0edf9ceb" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7a7bc2600743e1955b455be4056545"
I20260807 04:09:30.508147 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.486299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:45141 (local address 127.18.193.2:43837)
0807 04:09:30.500108 (+ 13809us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.500115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.500145 (+ 30us) server_negotiation.cc:407] Connection header received
0807 04:09:30.500228 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.500233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.500303 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.500433 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:30.500932 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.502088 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.504957 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.506336 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.506580 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.506685 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.507604 (+ 919us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.507763 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13645,"thread_start_us":13478,"threads_started":1}
I20260807 04:09:30.508289 21204 raft_consensus.cc:3037] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:30.508957 21350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424b841def6a40688fef758d0edf9ceb" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" is_pre_election: true
I20260807 04:09:30.509269 21350 raft_consensus.cc:2454] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 0.
I20260807 04:09:30.509634 21352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424b841def6a40688fef758d0edf9ceb" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800"
I20260807 04:09:30.509830 21352 raft_consensus.cc:3037] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:30.511464 21204 raft_consensus.cc:2454] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 1.
I20260807 04:09:30.511901 21418 leader_election.cc:302] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, e87708af134a40cfbd9c0d62cd57bed5; no voters:
I20260807 04:09:30.512248 21569 raft_consensus.cc:2781] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:30.513025 21352 raft_consensus.cc:2454] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 1.
I20260807 04:09:30.513720 21569 raft_consensus.cc:686] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 1 LEADER]: Becoming Leader. State: Replica: e87708af134a40cfbd9c0d62cd57bed5, State: Running, Role: LEADER
I20260807 04:09:30.514178 21569 consensus_queue.cc:237] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:30.518151 21039 catalog_manager.cc:5495] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3). New cstate: current_term: 1 leader_uuid: "e87708af134a40cfbd9c0d62cd57bed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260807 04:09:30.583688 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.576670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.18.193.3:42245)
0807 04:09:30.577317 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.577322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.577343 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:09:30.577475 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.577481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.577616 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.577785 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:30.578294 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.579206 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.581167 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.581609 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.581718 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.582258 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.582907 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.583024 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.583204 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.583272 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":390,"threads_started":1}
I20260807 04:09:30.600033 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.594383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:55955 (local address 127.18.193.1:36461)
0807 04:09:30.594584 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.594591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.594748 (+ 157us) server_negotiation.cc:407] Connection header received
0807 04:09:30.595177 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.595182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.595307 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.595421 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:30.595963 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.596794 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.598175 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.598657 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.598804 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.599255 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.599362 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.599469 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.599619 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.599708 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:09:30.600090 21625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.593949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36461 (local address 127.18.193.3:55955)
0807 04:09:30.594628 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:30.594660 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:30.594878 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:30.595437 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:30.595441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:30.595454 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:30.595782 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.595792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.596935 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.596943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.598025 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.598037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.598841 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.598849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.598998 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.599064 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.599470 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.599485 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 04:09:30.599596 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 04:09:30.599733 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":676,"threads_started":1}
I20260807 04:09:30.609861 21352 raft_consensus.cc:1261] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Refusing update from remote peer e87708af134a40cfbd9c0d62cd57bed5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:30.610733 21204 raft_consensus.cc:1261] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Refusing update from remote peer e87708af134a40cfbd9c0d62cd57bed5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:30.611253 21620 consensus_queue.cc:1035] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:30.611958 21569 consensus_queue.cc:1035] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:30.620872 21589 mvcc.cc:205] Tried to move back new op lower bound from 7315766356411392000 to 7315766356030017536. Current Snapshot: MvccSnapshot[applied={T|T < 7315766356411392000}]
I20260807 04:09:30.652540 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.645750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:40707 (local address 127.18.193.3:42245)
0807 04:09:30.645942 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:30.645948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:30.646329 (+ 381us) server_negotiation.cc:407] Connection header received
0807 04:09:30.646634 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:30.646642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:30.646736 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:30.646854 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:30.647349 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.648461 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.650544 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.651126 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.651248 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.651724 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.651870 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.651928 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:30.652124 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 04:09:30.652203 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:09:30.652720 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:30.645700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:42245 (local address 127.18.193.1:40707)
0807 04:09:30.646210 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:30.646230 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:30.646436 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:30.646909 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:30.646914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:30.646928 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:30.647182 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.647201 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.648657 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.648668 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.650391 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:30.650408 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:30.651276 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:30.651282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:30.651459 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:30.651508 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:30.651997 (+ 489us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:30.652004 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:09:30.652101 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 04:09:30.652279 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":407,"threads_started":1}
W20260807 04:09:30.653929 21591 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 04:09:32.287405 21039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:32.287698 21039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:32.293268 21039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Sending DeleteTablet for 3 replicas of tablet 424b841def6a40688fef758d0edf9ceb
I20260807 04:09:32.294487 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet 424b841def6a40688fef758d0edf9ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:32 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:35116
I20260807 04:09:32.294435 21469 tablet_service.cc:1514] Processing DeleteTablet for tablet 424b841def6a40688fef758d0edf9ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:32 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:51420
I20260807 04:09:32.294597 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 424b841def6a40688fef758d0edf9ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:32 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38324
I20260807 04:09:32.301775 21659 tablet_replica.cc:330] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: stopping tablet replica
I20260807 04:09:32.302093 21660 tablet_replica.cc:330] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: stopping tablet replica
I20260807 04:09:32.302309 21659 raft_consensus.cc:2229] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:09:32.302379 21660 raft_consensus.cc:2229] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:32.302810 21660 raft_consensus.cc:2258] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:32.303884 21661 tablet_replica.cc:330] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: stopping tablet replica
I20260807 04:09:32.303879 21659 raft_consensus.cc:2258] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:32.304162 21661 raft_consensus.cc:2229] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:32.304471 21660 ts_tablet_manager.cc:1905] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:32.304584 21661 raft_consensus.cc:2258] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:32.305447 21659 ts_tablet_manager.cc:1905] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:32.305660 21661 ts_tablet_manager.cc:1905] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:32.313970 21661 ts_tablet_manager.cc:1918] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:09:32.314132 21661 log.cc:1196] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/424b841def6a40688fef758d0edf9ceb
I20260807 04:09:32.314330 21660 ts_tablet_manager.cc:1918] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:09:32.314494 21660 log.cc:1196] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/424b841def6a40688fef758d0edf9ceb
I20260807 04:09:32.314669 21661 ts_tablet_manager.cc:1939] T 424b841def6a40688fef758d0edf9ceb P 0f7a7bc2600743e1955b455be4056545: Deleting consensus metadata
I20260807 04:09:32.315002 21660 ts_tablet_manager.cc:1939] T 424b841def6a40688fef758d0edf9ceb P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting consensus metadata
I20260807 04:09:32.315912 21021 catalog_manager.cc:4841] TS 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461): tablet 424b841def6a40688fef758d0edf9ceb (table test-table [id=9cfac104685f47578f8a681042895118]) successfully deleted
I20260807 04:09:32.316267 21021 catalog_manager.cc:4841] TS 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837): tablet 424b841def6a40688fef758d0edf9ceb (table test-table [id=9cfac104685f47578f8a681042895118]) successfully deleted
I20260807 04:09:32.323474 21659 ts_tablet_manager.cc:1918] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:09:32.323645 21659 log.cc:1196] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/424b841def6a40688fef758d0edf9ceb
I20260807 04:09:32.324184 21659 ts_tablet_manager.cc:1939] T 424b841def6a40688fef758d0edf9ceb P e87708af134a40cfbd9c0d62cd57bed5: Deleting consensus metadata
I20260807 04:09:32.325413 21022 catalog_manager.cc:4841] TS e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245): tablet 424b841def6a40688fef758d0edf9ceb (table test-table [id=9cfac104685f47578f8a681042895118]) successfully deleted
Aug 07 04:09:32 dist-test-slave-8n49 krb5kdc[20991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075772, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:09:32 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075772, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:32.366194 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.347680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.18.193.62:34981)
0807 04:09:32.348339 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:32.348344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:32.348657 (+ 313us) server_negotiation.cc:407] Connection header received
0807 04:09:32.348851 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:32.348855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:32.348941 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:32.349059 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:32.350534 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.351355 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.353073 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.353514 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.353647 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.358990 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:32.358995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:32.358999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:32.359047 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:32.362714 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:32.363627 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:32.363632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:32.363635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:32.363850 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:32.364297 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:32.364300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:32.364303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:32.364695 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:32.364901 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:32.365360 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 04:09:32.365575 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":341,"threads_started":1}
I20260807 04:09:32.371006 21039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:32.371805 21039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:32.379673 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 64dcb82c5ad54b2ba8fefe0b06eff245 (DEFAULT_TABLE table=test-table [id=171e5b0b004546749644314ed269410b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:32.379673 21469 tablet_service.cc:1467] Processing CreateTablet for tablet 64dcb82c5ad54b2ba8fefe0b06eff245 (DEFAULT_TABLE table=test-table [id=171e5b0b004546749644314ed269410b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:32.380167 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64dcb82c5ad54b2ba8fefe0b06eff245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:32.380167 21469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64dcb82c5ad54b2ba8fefe0b06eff245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:32.380110 21181 tablet_service.cc:1467] Processing CreateTablet for tablet 64dcb82c5ad54b2ba8fefe0b06eff245 (DEFAULT_TABLE table=test-table [id=171e5b0b004546749644314ed269410b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:32.380944 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64dcb82c5ad54b2ba8fefe0b06eff245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:32.388023 21670 tablet_bootstrap.cc:492] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: Bootstrap starting.
I20260807 04:09:32.388943 21671 tablet_bootstrap.cc:492] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: Bootstrap starting.
I20260807 04:09:32.389801 21672 tablet_bootstrap.cc:492] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: Bootstrap starting.
I20260807 04:09:32.391104 21671 tablet_bootstrap.cc:654] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:32.391353 21670 tablet_bootstrap.cc:654] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:32.392673 21672 tablet_bootstrap.cc:654] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:32.394353 21671 tablet_bootstrap.cc:492] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: No bootstrap required, opened a new log
I20260807 04:09:32.394569 21671 ts_tablet_manager.cc:1397] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260807 04:09:32.395643 21672 tablet_bootstrap.cc:492] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: No bootstrap required, opened a new log
I20260807 04:09:32.395445 21671 raft_consensus.cc:348] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.395763 21671 raft_consensus.cc:374] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:32.395851 21672 ts_tablet_manager.cc:1397] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260807 04:09:32.395870 21671 raft_consensus.cc:729] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7a7bc2600743e1955b455be4056545, State: Initialized, Role: FOLLOWER
I20260807 04:09:32.396162 21671 consensus_queue.cc:260] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.396835 21671 ts_tablet_manager.cc:1428] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 04:09:32.396750 21672 raft_consensus.cc:348] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.397054 21672 raft_consensus.cc:374] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:32.397158 21670 tablet_bootstrap.cc:492] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: No bootstrap required, opened a new log
I20260807 04:09:32.397270 21672 raft_consensus.cc:729] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87708af134a40cfbd9c0d62cd57bed5, State: Initialized, Role: FOLLOWER
I20260807 04:09:32.397398 21670 ts_tablet_manager.cc:1397] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260807 04:09:32.397607 21672 consensus_queue.cc:260] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.398500 21672 ts_tablet_manager.cc:1428] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 04:09:32.398736 21670 raft_consensus.cc:348] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.399046 21670 raft_consensus.cc:374] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:32.399230 21670 raft_consensus.cc:729] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8aa6bbbbea48ba9f45add94f6fe800, State: Initialized, Role: FOLLOWER
I20260807 04:09:32.399537 21670 consensus_queue.cc:260] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.401521 21670 ts_tablet_manager.cc:1428] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 04:09:32.501238 21636 raft_consensus.cc:482] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:32.501498 21636 raft_consensus.cc:504] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.502321 21636 leader_election.cc:288] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461), 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837)
I20260807 04:09:32.503042 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64dcb82c5ad54b2ba8fefe0b06eff245" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7a7bc2600743e1955b455be4056545" is_pre_election: true
I20260807 04:09:32.503204 21352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64dcb82c5ad54b2ba8fefe0b06eff245" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" is_pre_election: true
I20260807 04:09:32.503361 21204 raft_consensus.cc:2454] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 0.
I20260807 04:09:32.503623 21352 raft_consensus.cc:2454] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 0.
I20260807 04:09:32.504019 21418 leader_election.cc:302] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, e87708af134a40cfbd9c0d62cd57bed5; no voters:
I20260807 04:09:32.504317 21636 raft_consensus.cc:2781] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:32.504480 21636 raft_consensus.cc:482] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:32.504701 21636 raft_consensus.cc:3037] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:32.507175 21636 raft_consensus.cc:504] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.508325 21636 leader_election.cc:288] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 election: Requested vote from peers 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461), 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837)
I20260807 04:09:32.508432 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64dcb82c5ad54b2ba8fefe0b06eff245" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7a7bc2600743e1955b455be4056545"
I20260807 04:09:32.508747 21204 raft_consensus.cc:3037] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:32.508817 21352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64dcb82c5ad54b2ba8fefe0b06eff245" candidate_uuid: "e87708af134a40cfbd9c0d62cd57bed5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800"
I20260807 04:09:32.509156 21352 raft_consensus.cc:3037] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:32.511287 21204 raft_consensus.cc:2454] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 1.
I20260807 04:09:32.511569 21352 raft_consensus.cc:2454] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87708af134a40cfbd9c0d62cd57bed5 in term 1.
I20260807 04:09:32.511801 21418 leader_election.cc:302] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, e87708af134a40cfbd9c0d62cd57bed5; no voters:
I20260807 04:09:32.512151 21636 raft_consensus.cc:2781] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:32.512392 21636 raft_consensus.cc:686] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 LEADER]: Becoming Leader. State: Replica: e87708af134a40cfbd9c0d62cd57bed5, State: Running, Role: LEADER
I20260807 04:09:32.512851 21636 consensus_queue.cc:237] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.516279 21036 catalog_manager.cc:5495] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3). New cstate: current_term: 1 leader_uuid: "e87708af134a40cfbd9c0d62cd57bed5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260807 04:09:32.560220 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.553670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51438 (local address 127.18.193.3:42245)
0807 04:09:32.554634 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:32.554638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:32.554657 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:09:32.554711 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:32.554715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:32.554773 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:32.554855 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:32.555426 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.556450 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.558301 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.558739 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.558842 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.559383 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:32.559522 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:32.559599 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:32.559785 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 04:09:32.559847 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":677,"threads_started":1}
I20260807 04:09:32.571419 21352 raft_consensus.cc:1261] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Refusing update from remote peer e87708af134a40cfbd9c0d62cd57bed5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:32.571516 21204 raft_consensus.cc:1261] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Refusing update from remote peer e87708af134a40cfbd9c0d62cd57bed5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:32.572088 21636 consensus_queue.cc:1035] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:32.572479 21654 consensus_queue.cc:1035] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:32.577374 21641 mvcc.cc:205] Tried to move back new op lower bound from 7315766364447133696 to 7315766364215279616. Current Snapshot: MvccSnapshot[applied={T|T < 7315766364447133696}]
I20260807 04:09:32.626721 21036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:32.627007 21036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:32.630254 21036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Sending DeleteTablet for 3 replicas of tablet 64dcb82c5ad54b2ba8fefe0b06eff245
I20260807 04:09:32.631340 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet 64dcb82c5ad54b2ba8fefe0b06eff245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:32 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:35116
I20260807 04:09:32.631314 21469 tablet_service.cc:1514] Processing DeleteTablet for tablet 64dcb82c5ad54b2ba8fefe0b06eff245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:32 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:51420
I20260807 04:09:32.631484 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 64dcb82c5ad54b2ba8fefe0b06eff245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:32 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38324
I20260807 04:09:32.631708 21661 tablet_replica.cc:330] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: stopping tablet replica
I20260807 04:09:32.631789 21659 tablet_replica.cc:330] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: stopping tablet replica
I20260807 04:09:32.631918 21660 tablet_replica.cc:330] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: stopping tablet replica
I20260807 04:09:32.631942 21661 raft_consensus.cc:2229] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:32.632019 21659 raft_consensus.cc:2229] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:09:32.632169 21661 raft_consensus.cc:2258] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:32.632189 21660 raft_consensus.cc:2229] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:32.632354 21659 raft_consensus.cc:2258] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:32.632457 21660 raft_consensus.cc:2258] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:32.633459 21659 ts_tablet_manager.cc:1905] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:32.633456 21661 ts_tablet_manager.cc:1905] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:32.633728 21660 ts_tablet_manager.cc:1905] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:32.633677 21036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:32.634665 21036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:32.641840 21659 ts_tablet_manager.cc:1918] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:32.642035 21659 log.cc:1196] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/64dcb82c5ad54b2ba8fefe0b06eff245
I20260807 04:09:32.642228 21661 ts_tablet_manager.cc:1918] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:32.642478 21661 log.cc:1196] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/64dcb82c5ad54b2ba8fefe0b06eff245
I20260807 04:09:32.642524 21659 ts_tablet_manager.cc:1939] T 64dcb82c5ad54b2ba8fefe0b06eff245 P e87708af134a40cfbd9c0d62cd57bed5: Deleting consensus metadata
I20260807 04:09:32.643093 21661 ts_tablet_manager.cc:1939] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 0f7a7bc2600743e1955b455be4056545: Deleting consensus metadata
I20260807 04:09:32.644186 21469 tablet_service.cc:1467] Processing CreateTablet for tablet d09aea67ea5d4d46a393e9182d227416 (DEFAULT_TABLE table=test-table [id=4ecb51711c3844478521e0540b0d42d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:32.644214 21181 tablet_service.cc:1467] Processing CreateTablet for tablet d09aea67ea5d4d46a393e9182d227416 (DEFAULT_TABLE table=test-table [id=4ecb51711c3844478521e0540b0d42d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:32.644786 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d09aea67ea5d4d46a393e9182d227416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:32.644735 21022 catalog_manager.cc:4841] TS e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245): tablet 64dcb82c5ad54b2ba8fefe0b06eff245 (table test-table [id=171e5b0b004546749644314ed269410b]) successfully deleted
I20260807 04:09:32.644845 21469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d09aea67ea5d4d46a393e9182d227416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:32.644186 21325 tablet_service.cc:1467] Processing CreateTablet for tablet d09aea67ea5d4d46a393e9182d227416 (DEFAULT_TABLE table=test-table [id=4ecb51711c3844478521e0540b0d42d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:32.645268 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d09aea67ea5d4d46a393e9182d227416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:32.651922 21671 tablet_bootstrap.cc:492] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: Bootstrap starting.
I20260807 04:09:32.652401 21672 tablet_bootstrap.cc:492] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: Bootstrap starting.
I20260807 04:09:32.643455 21660 ts_tablet_manager.cc:1918] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:09:32.653198 21660 log.cc:1196] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/64dcb82c5ad54b2ba8fefe0b06eff245
I20260807 04:09:32.653957 21671 tablet_bootstrap.cc:654] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:32.653916 21660 ts_tablet_manager.cc:1939] T 64dcb82c5ad54b2ba8fefe0b06eff245 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting consensus metadata
I20260807 04:09:32.655010 21672 tablet_bootstrap.cc:654] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:32.655256 21021 catalog_manager.cc:4841] TS 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837): tablet 64dcb82c5ad54b2ba8fefe0b06eff245 (table test-table [id=171e5b0b004546749644314ed269410b]) successfully deleted
I20260807 04:09:32.655927 21670 tablet_bootstrap.cc:492] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: Bootstrap starting.
I20260807 04:09:32.656523 21671 tablet_bootstrap.cc:492] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: No bootstrap required, opened a new log
I20260807 04:09:32.656778 21671 ts_tablet_manager.cc:1397] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 04:09:32.658166 21672 tablet_bootstrap.cc:492] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: No bootstrap required, opened a new log
I20260807 04:09:32.658228 21670 tablet_bootstrap.cc:654] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:32.658442 21672 ts_tablet_manager.cc:1397] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.005s
I20260807 04:09:32.658459 21671 raft_consensus.cc:348] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.658593 21021 catalog_manager.cc:4841] TS 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461): tablet 64dcb82c5ad54b2ba8fefe0b06eff245 (table test-table [id=171e5b0b004546749644314ed269410b]) successfully deleted
I20260807 04:09:32.658846 21671 raft_consensus.cc:374] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:32.659046 21671 raft_consensus.cc:729] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7a7bc2600743e1955b455be4056545, State: Initialized, Role: FOLLOWER
I20260807 04:09:32.659387 21671 consensus_queue.cc:260] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.659788 21672 raft_consensus.cc:348] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.660090 21671 ts_tablet_manager.cc:1428] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 04:09:32.660401 21672 raft_consensus.cc:374] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:32.660514 21672 raft_consensus.cc:729] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87708af134a40cfbd9c0d62cd57bed5, State: Initialized, Role: FOLLOWER
I20260807 04:09:32.660802 21672 consensus_queue.cc:260] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.661433 21672 ts_tablet_manager.cc:1428] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 04:09:32.661909 21670 tablet_bootstrap.cc:492] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: No bootstrap required, opened a new log
I20260807 04:09:32.662184 21670 ts_tablet_manager.cc:1397] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260807 04:09:32.663885 21670 raft_consensus.cc:348] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.664271 21670 raft_consensus.cc:374] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:32.664382 21670 raft_consensus.cc:729] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8aa6bbbbea48ba9f45add94f6fe800, State: Initialized, Role: FOLLOWER
I20260807 04:09:32.664793 21670 consensus_queue.cc:260] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.665710 21670 ts_tablet_manager.cc:1428] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:32.825302 21676 raft_consensus.cc:482] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:32.825636 21676 raft_consensus.cc:504] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.827530 21676 leader_election.cc:288] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461), e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245)
I20260807 04:09:32.837451 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.828048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:48741 (local address 127.18.193.3:42245)
0807 04:09:32.828375 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:32.828385 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:32.830291 (+ 1906us) server_negotiation.cc:407] Connection header received
0807 04:09:32.830453 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:32.830460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:32.830578 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:32.830744 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:32.831788 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.832766 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.835215 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.836376 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.836586 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.836680 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:32.837004 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 04:09:32.837144 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 04:09:32.837702 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.827922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:42245 (local address 127.18.193.2:48741)
0807 04:09:32.830124 (+ 2202us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:32.830144 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:32.830308 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:32.830765 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:32.830769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:32.830794 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:32.831523 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.831535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.832959 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.832963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.835061 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.835074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.836640 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.836647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.836816 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.836841 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:09:32.836975 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 04:09:32.837132 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1971,"thread_start_us":910,"threads_started":1}
I20260807 04:09:32.838129 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.827736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36461 (local address 127.18.193.2:33205)
0807 04:09:32.829388 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:32.829418 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:32.829633 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:32.831045 (+ 1412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:32.831051 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:32.831095 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:32.831750 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.831767 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.833381 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.833389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.835232 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.835249 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.836910 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.836917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.837165 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.837190 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:09:32.837318 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 04:09:32.837462 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1448,"thread_start_us":1859,"threads_started":1}
I20260807 04:09:32.838485 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.828173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:33205 (local address 127.18.193.1:36461)
0807 04:09:32.830646 (+ 2473us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:32.830655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:32.830706 (+ 51us) server_negotiation.cc:407] Connection header received
0807 04:09:32.830785 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:32.830790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:32.830875 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:32.831012 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:32.831991 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.833222 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.835399 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.836544 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.836844 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.836944 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:32.837345 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 04:09:32.837490 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2196,"thread_start_us":1896,"threads_started":1}
I20260807 04:09:32.838456 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09aea67ea5d4d46a393e9182d227416" candidate_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87708af134a40cfbd9c0d62cd57bed5" is_pre_election: true
I20260807 04:09:32.838817 21489 raft_consensus.cc:2454] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b8aa6bbbbea48ba9f45add94f6fe800 in term 0.
I20260807 04:09:32.839478 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09aea67ea5d4d46a393e9182d227416" candidate_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7a7bc2600743e1955b455be4056545" is_pre_election: true
I20260807 04:09:32.839787 21204 raft_consensus.cc:2454] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b8aa6bbbbea48ba9f45add94f6fe800 in term 0.
I20260807 04:09:32.840337 21274 leader_election.cc:302] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, 1b8aa6bbbbea48ba9f45add94f6fe800; no voters:
I20260807 04:09:32.840826 21676 raft_consensus.cc:2781] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:32.841004 21676 raft_consensus.cc:482] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:32.841219 21676 raft_consensus.cc:3037] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:32.844702 21676 raft_consensus.cc:504] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.845486 21676 leader_election.cc:288] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [CANDIDATE]: Term 1 election: Requested vote from peers 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461), e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245)
I20260807 04:09:32.846035 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09aea67ea5d4d46a393e9182d227416" candidate_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f7a7bc2600743e1955b455be4056545"
I20260807 04:09:32.846330 21204 raft_consensus.cc:3037] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:32.846262 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d09aea67ea5d4d46a393e9182d227416" candidate_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87708af134a40cfbd9c0d62cd57bed5"
I20260807 04:09:32.846510 21489 raft_consensus.cc:3037] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:32.849735 21204 raft_consensus.cc:2454] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b8aa6bbbbea48ba9f45add94f6fe800 in term 1.
I20260807 04:09:32.849943 21489 raft_consensus.cc:2454] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b8aa6bbbbea48ba9f45add94f6fe800 in term 1.
I20260807 04:09:32.850258 21274 leader_election.cc:302] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f7a7bc2600743e1955b455be4056545, 1b8aa6bbbbea48ba9f45add94f6fe800; no voters:
I20260807 04:09:32.850540 21676 raft_consensus.cc:2781] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:32.851006 21676 raft_consensus.cc:686] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 LEADER]: Becoming Leader. State: Replica: 1b8aa6bbbbea48ba9f45add94f6fe800, State: Running, Role: LEADER
I20260807 04:09:32.851390 21676 consensus_queue.cc:237] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } }
I20260807 04:09:32.855345 21036 catalog_manager.cc:5495] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2). New cstate: current_term: 1 leader_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b8aa6bbbbea48ba9f45add94f6fe800" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 43837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260807 04:09:32.915077 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.907280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.18.193.2:43837)
0807 04:09:32.908153 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:32.908157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:32.908176 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:09:32.908261 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:32.908265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:32.908338 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:32.908533 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:32.909215 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.910234 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.911874 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.912569 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.912717 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.913482 (+ 765us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:32.914197 (+ 715us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:32.914360 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:32.914512 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 04:09:32.914596 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":485,"threads_started":1}
I20260807 04:09:32.935014 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.926770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:57703 (local address 127.18.193.1:36461)
0807 04:09:32.927022 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:32.927037 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:32.928138 (+ 1101us) server_negotiation.cc:407] Connection header received
0807 04:09:32.928309 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:32.928314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:32.928377 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:32.928450 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:32.929354 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.930376 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.932730 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.933347 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.933455 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.934049 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:32.934158 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:32.934243 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:32.934462 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 04:09:32.934533 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:09:32.935192 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.926887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36461 (local address 127.18.193.2:57703)
0807 04:09:32.928038 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:32.928054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:32.928185 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:32.928480 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:32.928483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:32.928493 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:32.929215 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.929226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.930597 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.930601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.932481 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.932494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.933612 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.933618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.933778 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.933836 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:32.934295 (+ 459us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:32.934307 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 04:09:32.934390 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 04:09:32.934534 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"thread_start_us":673,"threads_started":1}
I20260807 04:09:32.945439 21489 raft_consensus.cc:1261] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Refusing update from remote peer 1b8aa6bbbbea48ba9f45add94f6fe800: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:32.945638 21204 raft_consensus.cc:1261] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Refusing update from remote peer 1b8aa6bbbbea48ba9f45add94f6fe800: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:32.946483 21691 consensus_queue.cc:1035] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e87708af134a40cfbd9c0d62cd57bed5" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:32.946926 21676 consensus_queue.cc:1035] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f7a7bc2600743e1955b455be4056545" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:32.975821 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.970192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:36761 (local address 127.18.193.2:43837)
0807 04:09:32.970386 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:32.970392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:32.970413 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:09:32.970497 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:32.970502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:32.970581 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:32.970683 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:32.971234 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.971987 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.973760 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.974244 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.974395 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.974942 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:32.975044 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:32.975148 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:32.975297 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 04:09:32.975386 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:09:32.975915 21701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:32.969391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:43837 (local address 127.18.193.1:36761)
0807 04:09:32.970162 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:32.970185 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:32.970373 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:32.970701 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:32.970705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:32.970717 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:09:32.971063 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.971074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.972134 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.972140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.973610 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:32.973623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:32.974518 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:32.974524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:32.974670 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:32.974708 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:09:32.975154 (+ 446us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:09:32.975168 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 04:09:32.975271 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 04:09:32.975425 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":387,"threads_started":1}
W20260807 04:09:32.977094 21640 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 04:09:34.580725 21038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:34.581197 21038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:34.587107 21038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a68856f2b0824b31be4e6f485aa4263b: Sending DeleteTablet for 3 replicas of tablet d09aea67ea5d4d46a393e9182d227416
I20260807 04:09:34.588294 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet d09aea67ea5d4d46a393e9182d227416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:34 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38324
I20260807 04:09:34.588325 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet d09aea67ea5d4d46a393e9182d227416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:34 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:35116
I20260807 04:09:34.588692 21469 tablet_service.cc:1514] Processing DeleteTablet for tablet d09aea67ea5d4d46a393e9182d227416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:34 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:51420
I20260807 04:09:34.589962 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21113
I20260807 04:09:34.604655 21728 tablet_replica.cc:330] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: stopping tablet replica
I20260807 04:09:34.605155 21728 raft_consensus.cc:2229] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:34.605782 21728 raft_consensus.cc:2258] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:34.607898 21728 ts_tablet_manager.cc:1905] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:34.611850 21725 tablet_replica.cc:330] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: stopping tablet replica
I20260807 04:09:34.612097 21725 raft_consensus.cc:2229] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:34.612396 21725 raft_consensus.cc:2258] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:34.613778 21725 ts_tablet_manager.cc:1905] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:34.614806 21727 tablet_replica.cc:330] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: stopping tablet replica
I20260807 04:09:34.615098 21727 raft_consensus.cc:2229] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:09:34.615968 21727 raft_consensus.cc:2258] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:34.618121 21728 ts_tablet_manager.cc:1918] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:09:34.618384 21728 log.cc:1196] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/d09aea67ea5d4d46a393e9182d227416
I20260807 04:09:34.618783 21727 ts_tablet_manager.cc:1905] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:34.619210 21728 ts_tablet_manager.cc:1939] T d09aea67ea5d4d46a393e9182d227416 P e87708af134a40cfbd9c0d62cd57bed5: Deleting consensus metadata
I20260807 04:09:34.620471 21022 catalog_manager.cc:4841] TS e87708af134a40cfbd9c0d62cd57bed5 (127.18.193.3:42245): tablet d09aea67ea5d4d46a393e9182d227416 (table test-table [id=4ecb51711c3844478521e0540b0d42d3]) successfully deleted
I20260807 04:09:34.624658 21725 ts_tablet_manager.cc:1918] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:09:34.625085 21725 log.cc:1196] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/d09aea67ea5d4d46a393e9182d227416
Aug 07 04:09:34 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075772, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:34.626084 21725 ts_tablet_manager.cc:1939] T d09aea67ea5d4d46a393e9182d227416 P 0f7a7bc2600743e1955b455be4056545: Deleting consensus metadata
I20260807 04:09:34.627606 21021 catalog_manager.cc:4841] TS 0f7a7bc2600743e1955b455be4056545 (127.18.193.1:36461): tablet d09aea67ea5d4d46a393e9182d227416 (table test-table [id=4ecb51711c3844478521e0540b0d42d3]) successfully deleted
I20260807 04:09:34.628895 21727 ts_tablet_manager.cc:1918] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:09:34.629084 21727 log.cc:1196] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/d09aea67ea5d4d46a393e9182d227416
I20260807 04:09:34.629583 21727 ts_tablet_manager.cc:1939] T d09aea67ea5d4d46a393e9182d227416 P 1b8aa6bbbbea48ba9f45add94f6fe800: Deleting consensus metadata
I20260807 04:09:34.630813 21021 catalog_manager.cc:4841] TS 1b8aa6bbbbea48ba9f45add94f6fe800 (127.18.193.2:43837): tablet d09aea67ea5d4d46a393e9182d227416 (table test-table [id=4ecb51711c3844478521e0540b0d42d3]) successfully deleted
I20260807 04:09:34.635272 21730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:34.592037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.18.193.1:36461)
0807 04:09:34.605225 (+ 13188us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:34.605236 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:34.605278 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:09:34.605374 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:34.605384 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:34.605525 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:34.608309 (+ 2784us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:34.608653 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:34.610353 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:34.612923 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:34.614544 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:34.614689 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:34.627556 (+ 12867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:34.627581 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:34.627606 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:34.627662 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:34.631900 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:34.632795 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:34.632803 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:34.632805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:34.633018 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:34.633468 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:34.633472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:34.633474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:34.633875 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:34.634199 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:34.634594 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 04:09:34.634838 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12992,"thread_start_us":12828,"threads_started":1}
W20260807 04:09:34.636191 21249 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35132
W20260807 04:09:34.637001 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21113: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:34.637236 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 21113
I20260807 04:09:34.668138 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21258
Aug 07 04:09:34 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075772, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:09:34.684396 21731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:34.668945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.18.193.2:43837)
0807 04:09:34.669521 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:34.669525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:34.669540 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:09:34.669596 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:34.669599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:34.669674 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:34.669801 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:34.671314 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:34.672086 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:34.673472 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:34.673989 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:34.674131 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:34.678374 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:34.678390 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:34.678408 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:34.678443 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:34.681506 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:34.682219 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:34.682227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:34.682229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:34.682361 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:34.682767 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:34.682773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:34.682776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:34.683169 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:34.683498 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:34.683797 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 04:09:34.684020 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":272,"threads_started":1}
W20260807 04:09:34.685124 21393 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38344
W20260807 04:09:34.685776 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21258: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:34.685916 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 21258
I20260807 04:09:34.713268 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21402
Aug 07 04:09:34 dist-test-slave-8n49 krb5kdc[20991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075772, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:09:34.730146 21732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:34.714085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.18.193.3:42245)
0807 04:09:34.714858 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:34.714862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:34.714879 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:09:34.714935 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:34.714938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:34.715009 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:34.715172 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:34.717143 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:34.717920 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:34.719100 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:34.719521 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:34.719622 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:34.723924 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:34.723946 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:34.723968 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:34.724012 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:34.727235 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:34.727927 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:34.727934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:34.727937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:34.728073 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:34.728444 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:34.728447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:34.728450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:34.728942 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:34.729199 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:34.729499 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 04:09:34.729714 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":472,"threads_started":1}
W20260807 04:09:34.731146 21536 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51450
W20260807 04:09:34.731995 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21402: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:34.732250 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 21402
I20260807 04:09:34.759182 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21004
I20260807 04:09:34.760680 21081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:34.882148 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 21004
2026-08-07T04:09:34Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9123 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:34 dist-test-slave-8n49 krb5kdc[21736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:34 dist-test-slave-8n49 krb5kdc[21736](info): set up 2 sockets
Aug 07 04:09:34 dist-test-slave-8n49 krb5kdc[21736](info): commencing operation
krb5kdc: starting...
W20260807 04:09:36.040045 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:36 dist-test-slave-8n49 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:36.262120 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37895
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:37895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:36.612268 21749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:36.612653 21749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:36.612819 21749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:36.620182 21749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:36.620289 21749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:36.620368 21749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:36.620457 21749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:36.620507 21749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:36.630671 21749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:37895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:36.632035 21749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:36.633461 21749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:36.645341 21754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:36.645376 21757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:36.645303 21755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:36.646492 21749 server_base.cc:1034] running on GCE node
I20260807 04:09:36.647255 21749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:36.648747 21749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:36.649993 21749 hybrid_clock.cc:648] HybridClock initialized: now 1786075776649948 us; error 31 us; skew 500 ppm
I20260807 04:09:36.650380 21749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:36 dist-test-slave-8n49 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075776, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:36.653836 21749 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:36.655407 21749 webserver.cc:460] Webserver started at http://127.18.193.62:41631/ using document root <none> and password file <none>
I20260807 04:09:36.656131 21749 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:36.656258 21749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:36.656649 21749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:36.659646 21749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "54c0aecfba2242359973dfd56e9f4ae0"
format_stamp: "Formatted at 2026-08-07 04:09:36 on dist-test-slave-8n49"
server_key: "170f5f7e75d23932f5cbf13625a89453"
server_key_iv: "eb8eba284bc75241d38f1cd912c39b56"
server_key_version: "encryptionkey@0"
I20260807 04:09:36.660440 21749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "54c0aecfba2242359973dfd56e9f4ae0"
format_stamp: "Formatted at 2026-08-07 04:09:36 on dist-test-slave-8n49"
server_key: "170f5f7e75d23932f5cbf13625a89453"
server_key_iv: "eb8eba284bc75241d38f1cd912c39b56"
server_key_version: "encryptionkey@0"
I20260807 04:09:36.665623 21749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:09:36.669157 21764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:36.670506 21749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:09:36.670697 21749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "54c0aecfba2242359973dfd56e9f4ae0"
format_stamp: "Formatted at 2026-08-07 04:09:36 on dist-test-slave-8n49"
server_key: "170f5f7e75d23932f5cbf13625a89453"
server_key_iv: "eb8eba284bc75241d38f1cd912c39b56"
server_key_version: "encryptionkey@0"
I20260807 04:09:36.670904 21749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:36.687812 21749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:36.692703 21749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:36.693005 21749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:36.713001 21749 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:37895
I20260807 04:09:36.713032 21818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:37895 every 8 connection(s)
I20260807 04:09:36.714489 21749 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:36.718495 21819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:36.724509 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 21749
I20260807 04:09:36.724804 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:36.725636 19204 external_mini_cluster.cc:1428] Setting key 3d2575545ff81318dfe1db1c0f82be79
I20260807 04:09:36.730764 21819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0: Bootstrap starting.
I20260807 04:09:36.734081 21819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:36.735179 21819 log.cc:824] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:36.738538 21819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0: No bootstrap required, opened a new log
Aug 07 04:09:36 dist-test-slave-8n49 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:36.748948 21819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0aecfba2242359973dfd56e9f4ae0" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37895 } }
I20260807 04:09:36.749455 21819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:36.749562 21819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c0aecfba2242359973dfd56e9f4ae0, State: Initialized, Role: FOLLOWER
I20260807 04:09:36.750365 21819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0aecfba2242359973dfd56e9f4ae0" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37895 } }
I20260807 04:09:36.750662 21819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:36.750811 21819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:36.751010 21819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:36.751803 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:36.728058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.18.193.62:37895)
0807 04:09:36.728955 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:36.728967 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:36.729008 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:09:36.730830 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:36.730878 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:36.731293 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:36.731669 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:36.733305 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:36.734694 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:36.736266 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:36.737297 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:36.737538 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:36.742170 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:36.742181 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:36.742207 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:36.742268 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:36.746681 (+ 4413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:36.747574 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:36.747584 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:36.747593 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:36.747830 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:36.748272 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:36.748277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:36.748280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:36.749114 (+ 834us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:36.749433 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:36.750064 (+ 631us) server_negotiation.cc:299] Negotiation successful
0807 04:09:36.750518 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":432,"threads_started":1}
I20260807 04:09:36.753971 21819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0aecfba2242359973dfd56e9f4ae0" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37895 } }
I20260807 04:09:36.754575 21819 leader_election.cc:302] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c0aecfba2242359973dfd56e9f4ae0; no voters:
I20260807 04:09:36.755357 21819 leader_election.cc:288] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:36.755467 21824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:36.756271 21824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [term 1 LEADER]: Becoming Leader. State: Replica: 54c0aecfba2242359973dfd56e9f4ae0, State: Running, Role: LEADER
I20260807 04:09:36.756909 21824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0aecfba2242359973dfd56e9f4ae0" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37895 } }
I20260807 04:09:36.757654 21819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:36.760159 21825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54c0aecfba2242359973dfd56e9f4ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0aecfba2242359973dfd56e9f4ae0" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37895 } } }
I20260807 04:09:36.760660 21825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:36.760169 21826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c0aecfba2242359973dfd56e9f4ae0. Latest consensus state: current_term: 1 leader_uuid: "54c0aecfba2242359973dfd56e9f4ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0aecfba2242359973dfd56e9f4ae0" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37895 } } }
I20260807 04:09:36.762660 21826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:36.764283 21830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:36.769845 21830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:36.779517 21830 catalog_manager.cc:1347] Generated new cluster ID: 2a3418c453104e339c790c0595726169
I20260807 04:09:36.779666 21830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:36.796914 21830 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:36.798298 21830 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:36.814496 21830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54c0aecfba2242359973dfd56e9f4ae0: Generated new TSK 0
I20260807 04:09:36.815632 21830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:36.884124 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37895
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:37.225768 21845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:37.226044 21845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:37.226229 21845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:37.233482 21845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:37.233630 21845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:37.233816 21845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:37.243794 21845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:37.245232 21845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:37.246577 21845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:37.260013 21850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:37.259964 21851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:37.260082 21853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:37.261394 21845 server_base.cc:1034] running on GCE node
I20260807 04:09:37.262153 21845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:37.263217 21845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:37.264442 21845 hybrid_clock.cc:648] HybridClock initialized: now 1786075777264374 us; error 60 us; skew 500 ppm
I20260807 04:09:37.264814 21845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:37 dist-test-slave-8n49 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075777, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:37.267937 21845 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:37.269483 21845 webserver.cc:460] Webserver started at http://127.18.193.1:41581/ using document root <none> and password file <none>
I20260807 04:09:37.270148 21845 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:37.270248 21845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:37.270524 21845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:37.273641 21845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "bc678fab47144c1b9d3bb0741649c85c"
format_stamp: "Formatted at 2026-08-07 04:09:37 on dist-test-slave-8n49"
server_key: "6fcc8951e04632b0c6b578dc674b66d4"
server_key_iv: "fcb9b9260c214dc7ca4253c823cb2fa5"
server_key_version: "encryptionkey@0"
I20260807 04:09:37.274513 21845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "bc678fab47144c1b9d3bb0741649c85c"
format_stamp: "Formatted at 2026-08-07 04:09:37 on dist-test-slave-8n49"
server_key: "6fcc8951e04632b0c6b578dc674b66d4"
server_key_iv: "fcb9b9260c214dc7ca4253c823cb2fa5"
server_key_version: "encryptionkey@0"
I20260807 04:09:37.279949 21845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:09:37.283596 21860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:37.284996 21845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:37.285184 21845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "bc678fab47144c1b9d3bb0741649c85c"
format_stamp: "Formatted at 2026-08-07 04:09:37 on dist-test-slave-8n49"
server_key: "6fcc8951e04632b0c6b578dc674b66d4"
server_key_iv: "fcb9b9260c214dc7ca4253c823cb2fa5"
server_key_version: "encryptionkey@0"
I20260807 04:09:37.285398 21845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:37.307886 21845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:37.312309 21845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:37.312670 21845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:37.313721 21845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:37.315340 21845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:37.315452 21845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:37.315605 21845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:37.315680 21845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:37.350023 21845 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:40213
I20260807 04:09:37.350121 21978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:40213 every 8 connection(s)
I20260807 04:09:37.351380 21845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:37.358503 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 21845
I20260807 04:09:37.358743 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:37.359462 19204 external_mini_cluster.cc:1428] Setting key 45e6a37bca6c189aec9f52f64d614cfe
Aug 07 04:09:37 dist-test-slave-8n49 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075777, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:37.375990 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:37.354090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:40687 (local address 127.18.193.62:37895)
0807 04:09:37.354572 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:37.354577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:37.355598 (+ 1021us) server_negotiation.cc:407] Connection header received
0807 04:09:37.357628 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:37.357633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:37.357734 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:37.357882 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:37.359631 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.360376 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.362181 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.362922 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.363088 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:37.367224 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:37.367233 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:37.367241 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:37.367288 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:37.370579 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:37.371356 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:37.371365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:37.371371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:37.371542 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:37.372324 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:37.372332 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:37.372338 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:37.372792 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:37.372952 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:37.375389 (+ 2437us) server_negotiation.cc:299] Negotiation successful
0807 04:09:37.375653 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":247,"threads_started":1}
I20260807 04:09:37.376386 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:37.354657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37895 (local address 127.18.193.1:40687)
0807 04:09:37.355406 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:37.355445 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:37.357338 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:37.358144 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:37.358163 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:37.358772 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:37.359444 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:37.359470 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.360644 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.360649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:37.362019 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:37.362033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.363179 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.363184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:37.363418 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:37.364199 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:37.364223 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:37.366965 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:37.370738 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:37.370748 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:37.370767 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:37.371217 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:37.371711 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:37.371721 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:37.371728 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:37.372154 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:37.372955 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:37.372965 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:37.373741 (+ 776us) client_negotiation.cc:769] Sending connection context
0807 04:09:37.374153 (+ 412us) client_negotiation.cc:240] Negotiation successful
0807 04:09:37.374727 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":299,"threads_started":1}
I20260807 04:09:37.378620 21979 heartbeater.cc:347] Connected to a master server at 127.18.193.62:37895
I20260807 04:09:37.379034 21979 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:37.379793 21979 heartbeater.cc:511] Master 127.18.193.62:37895 requested a full tablet report, sending...
I20260807 04:09:37.382434 21783 ts_manager.cc:194] Registered new tserver with Master: bc678fab47144c1b9d3bb0741649c85c (127.18.193.1:40213)
I20260807 04:09:37.384734 21783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:40687
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:37.416024 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37895
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:37.744988 21985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:37.745271 21985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:37.745445 21985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:37.752780 21985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:37.752894 21985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:37.753108 21985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:37.763053 21985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:37.764480 21985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:37.765916 21985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:37.779557 21990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:37.779630 21993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:37.779634 21991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:37.780685 21985 server_base.cc:1034] running on GCE node
I20260807 04:09:37.781342 21985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:37.782387 21985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:37.783622 21985 hybrid_clock.cc:648] HybridClock initialized: now 1786075777783552 us; error 69 us; skew 500 ppm
I20260807 04:09:37.783962 21985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:37 dist-test-slave-8n49 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075777, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:37.787202 21985 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:37.788877 21985 webserver.cc:460] Webserver started at http://127.18.193.2:43285/ using document root <none> and password file <none>
I20260807 04:09:37.789697 21985 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:37.789839 21985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:37.790158 21985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:37.793192 21985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "7a30dbbbedc446e0a01bef7997d136a4"
format_stamp: "Formatted at 2026-08-07 04:09:37 on dist-test-slave-8n49"
server_key: "237b718dd62aee9f8a20fd7f7fa83f09"
server_key_iv: "70e2d97222130fa6a1e56ea24d9ee199"
server_key_version: "encryptionkey@0"
I20260807 04:09:37.793982 21985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "7a30dbbbedc446e0a01bef7997d136a4"
format_stamp: "Formatted at 2026-08-07 04:09:37 on dist-test-slave-8n49"
server_key: "237b718dd62aee9f8a20fd7f7fa83f09"
server_key_iv: "70e2d97222130fa6a1e56ea24d9ee199"
server_key_version: "encryptionkey@0"
I20260807 04:09:37.799243 21985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:09:37.802857 22000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:37.804308 21985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:09:37.804499 21985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "7a30dbbbedc446e0a01bef7997d136a4"
format_stamp: "Formatted at 2026-08-07 04:09:37 on dist-test-slave-8n49"
server_key: "237b718dd62aee9f8a20fd7f7fa83f09"
server_key_iv: "70e2d97222130fa6a1e56ea24d9ee199"
server_key_version: "encryptionkey@0"
I20260807 04:09:37.804814 21985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:37.834529 21985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:37.839509 21985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:37.839864 21985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:37.840961 21985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:37.842473 21985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:37.842594 21985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:37.842756 21985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:37.842839 21985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:37.880631 21985 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:38093
I20260807 04:09:37.880693 22118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:38093 every 8 connection(s)
I20260807 04:09:37.881943 21985 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:37.887035 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 21985
I20260807 04:09:37.887239 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:37.887682 19204 external_mini_cluster.cc:1428] Setting key 09515ba7fc00c4b5a00ad75555821523
Aug 07 04:09:37 dist-test-slave-8n49 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075777, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:37.905593 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:37.884762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:39807 (local address 127.18.193.62:37895)
0807 04:09:37.885281 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:37.885286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:37.886169 (+ 883us) server_negotiation.cc:407] Connection header received
0807 04:09:37.888347 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:37.888354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:37.888452 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:37.888677 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:37.890425 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.891243 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.893016 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.893496 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.893650 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:37.897760 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:37.897770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:37.897779 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:37.897829 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:37.901932 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:37.902717 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:37.902726 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:37.902733 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:37.902911 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:37.903355 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:37.903363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:37.903369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:37.903787 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:37.903956 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:37.904695 (+ 739us) server_negotiation.cc:299] Negotiation successful
0807 04:09:37.905002 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":298,"threads_started":1}
I20260807 04:09:37.906141 22122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:37.885271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37895 (local address 127.18.193.2:39807)
0807 04:09:37.885996 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:37.886041 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:37.888058 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:37.888900 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:37.888910 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:37.889556 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:37.890246 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:37.890265 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.891406 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.891417 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:37.892856 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:37.892869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:37.893745 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:37.893755 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:37.894003 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:37.894890 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:37.894914 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:37.897498 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:37.902097 (+ 4599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:37.902104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:37.902118 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:37.902565 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:37.903031 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:37.903035 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:37.903037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:37.903224 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:37.903920 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:37.903926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:37.904262 (+ 336us) client_negotiation.cc:769] Sending connection context
0807 04:09:37.904495 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 04:09:37.904970 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":302,"threads_started":1}
I20260807 04:09:37.907884 22119 heartbeater.cc:347] Connected to a master server at 127.18.193.62:37895
I20260807 04:09:37.908216 22119 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:37.909111 22119 heartbeater.cc:511] Master 127.18.193.62:37895 requested a full tablet report, sending...
I20260807 04:09:37.910696 21783 ts_manager.cc:194] Registered new tserver with Master: 7a30dbbbedc446e0a01bef7997d136a4 (127.18.193.2:38093)
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
I20260807 04:09:37.911558 21783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:39807
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:37.940956 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37895
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:38.278198 22125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:38.278484 22125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:38.278638 22125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:38.286218 22125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:38.286322 22125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:38.286504 22125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:38.296756 22125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:38.298061 22125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:38.299403 22125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:38.313103 22131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:38.313150 22130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:38.313189 22133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:38.314083 22125 server_base.cc:1034] running on GCE node
I20260807 04:09:38.314684 22125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:38.315680 22125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:38.316951 22125 hybrid_clock.cc:648] HybridClock initialized: now 1786075778316895 us; error 55 us; skew 500 ppm
I20260807 04:09:38.317340 22125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:38 dist-test-slave-8n49 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075778, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:38.320792 22125 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:38.322495 22125 webserver.cc:460] Webserver started at http://127.18.193.3:44779/ using document root <none> and password file <none>
I20260807 04:09:38.323271 22125 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:38.323444 22125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:38.323792 22125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:38.327564 22125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "4899269882dc496baf803eeeaf2dd4e1"
format_stamp: "Formatted at 2026-08-07 04:09:38 on dist-test-slave-8n49"
server_key: "5252e34c7be65cd09f8bd24a2e647626"
server_key_iv: "e37c1898cb1971d6ecfe668ea87b6e8a"
server_key_version: "encryptionkey@0"
I20260807 04:09:38.328464 22125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "4899269882dc496baf803eeeaf2dd4e1"
format_stamp: "Formatted at 2026-08-07 04:09:38 on dist-test-slave-8n49"
server_key: "5252e34c7be65cd09f8bd24a2e647626"
server_key_iv: "e37c1898cb1971d6ecfe668ea87b6e8a"
server_key_version: "encryptionkey@0"
I20260807 04:09:38.334141 22125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:09:38.337970 22141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:38.339354 22125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:09:38.339532 22125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "4899269882dc496baf803eeeaf2dd4e1"
format_stamp: "Formatted at 2026-08-07 04:09:38 on dist-test-slave-8n49"
server_key: "5252e34c7be65cd09f8bd24a2e647626"
server_key_iv: "e37c1898cb1971d6ecfe668ea87b6e8a"
server_key_version: "encryptionkey@0"
I20260807 04:09:38.339777 22125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:38.366465 22125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:38.370818 22125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:38.371137 22125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:38.372212 22125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:38.373600 22125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:38.373708 22125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:38.373865 22125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:38.373939 22125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:38.387900 21979 heartbeater.cc:503] Master 127.18.193.62:37895 was elected leader, sending a full tablet report...
I20260807 04:09:38.408375 22125 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:44367
I20260807 04:09:38.408465 22258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:44367 every 8 connection(s)
I20260807 04:09:38.409837 22125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:38.412907 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 22125
I20260807 04:09:38.413141 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:38.413733 19204 external_mini_cluster.cc:1428] Setting key 7878c96651cc76fab5a1f860044e5c0c
Aug 07 04:09:38 dist-test-slave-8n49 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075778, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:38.430332 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:38.412299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:36585 (local address 127.18.193.62:37895)
0807 04:09:38.412888 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:38.412896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:38.413979 (+ 1083us) server_negotiation.cc:407] Connection header received
0807 04:09:38.416418 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:38.416423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:38.416527 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:38.416695 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:38.418248 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:38.418997 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:38.420529 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:38.420976 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:38.421075 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:38.424329 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:38.424334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:38.424337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:38.424367 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:38.426949 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:38.427834 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:38.427843 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:38.427848 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:38.428000 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:38.428542 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:38.428547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:38.428573 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:38.428862 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:38.429026 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:38.429730 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 04:09:38.429941 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":298,"threads_started":1}
I20260807 04:09:38.431262 22263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:38.412893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37895 (local address 127.18.193.3:36585)
0807 04:09:38.413783 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:38.413851 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:38.416150 (+ 2299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:38.416934 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:38.416949 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:38.417533 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:38.418086 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:38.418104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:38.419120 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:38.419124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:38.420394 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:38.420406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:38.421172 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:38.421177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:38.421364 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:38.422003 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:38.422024 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:38.424142 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:38.427080 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:38.427086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:38.427098 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:38.427684 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:38.428143 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:38.428149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:38.428152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:38.428421 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:38.429019 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:38.429026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:38.429355 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 04:09:38.429640 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 04:09:38.430163 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":351,"threads_started":1}
I20260807 04:09:38.432623 22259 heartbeater.cc:347] Connected to a master server at 127.18.193.62:37895
I20260807 04:09:38.432941 22259 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:38.433656 22259 heartbeater.cc:511] Master 127.18.193.62:37895 requested a full tablet report, sending...
I20260807 04:09:38.435096 21783 ts_manager.cc:194] Registered new tserver with Master: 4899269882dc496baf803eeeaf2dd4e1 (127.18.193.3:44367)
I20260807 04:09:38.435891 21783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:36585
I20260807 04:09:38.441645 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 04:09:38.487649 22261 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:09:38.484883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49874 (local address 127.18.193.62:37895)
0807 04:09:38.485136 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:38.485145 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:38.485523 (+ 378us) server_negotiation.cc:407] Connection header received
0807 04:09:38.485763 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:38.485767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:38.485858 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:38.485977 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:38.487392 (+ 1415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49874: BlockingRecv error: recv got EOF from 127.0.0.1:49874 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:09:38.491571 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21845
W20260807 04:09:38.494530 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21845: Not authorized: Client connection negotiation failed: client connection to 127.18.193.1:40213: server requires authentication, but client does not have Kerberos credentials available
I20260807 04:09:38.494761 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 21845
I20260807 04:09:38.512687 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21985
W20260807 04:09:38.515442 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21985: Not authorized: Client connection negotiation failed: client connection to 127.18.193.2:38093: server requires authentication, but client does not have Kerberos credentials available
I20260807 04:09:38.515609 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 21985
I20260807 04:09:38.532963 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22125
W20260807 04:09:38.536135 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22125: Not authorized: Client connection negotiation failed: client connection to 127.18.193.3:44367: server requires authentication, but client does not have Kerberos credentials available
I20260807 04:09:38.536316 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 22125
I20260807 04:09:38.553438 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 21749
I20260807 04:09:38.554827 21814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:38.661265 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 21749
2026-08-07T04:09:38Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3775 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:38 dist-test-slave-8n49 krb5kdc[22278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:38 dist-test-slave-8n49 krb5kdc[22278](info): set up 2 sockets
Aug 07 04:09:38 dist-test-slave-8n49 krb5kdc[22278](info): commencing operation
krb5kdc: starting...
W20260807 04:09:39.798739 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:39 dist-test-slave-8n49 krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:40.025831 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41119
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:41119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:40.413676 22291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:40.414178 22291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:40.414327 22291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:40.426885 22291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:40.427074 22291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:40.427147 22291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:40.427233 22291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:40.427299 22291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:40.445132 22291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:41119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:40.447340 22291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:40.449734 22291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:40.472610 22298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:40.472479 22301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:40.472534 22296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:40.473342 22291 server_base.cc:1034] running on GCE node
I20260807 04:09:40.475178 22291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:40.478015 22291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:40.479336 22291 hybrid_clock.cc:648] HybridClock initialized: now 1786075780479253 us; error 60 us; skew 500 ppm
I20260807 04:09:40.479972 22291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:40 dist-test-slave-8n49 krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075780, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:40.489543 22291 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:40.492144 22291 webserver.cc:460] Webserver started at http://127.18.193.62:36913/ using document root <none> and password file <none>
I20260807 04:09:40.493471 22291 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:40.493731 22291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:40.494349 22291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:40.499888 22291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "0c8b96d67d314cb382e9afca81f2c68f"
format_stamp: "Formatted at 2026-08-07 04:09:40 on dist-test-slave-8n49"
server_key: "38000595ed38227a4d27e4ee5b468ea1"
server_key_iv: "0822dc744620a401e99ccc8df76706d4"
server_key_version: "encryptionkey@0"
I20260807 04:09:40.501502 22291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "0c8b96d67d314cb382e9afca81f2c68f"
format_stamp: "Formatted at 2026-08-07 04:09:40 on dist-test-slave-8n49"
server_key: "38000595ed38227a4d27e4ee5b468ea1"
server_key_iv: "0822dc744620a401e99ccc8df76706d4"
server_key_version: "encryptionkey@0"
I20260807 04:09:40.510622 22291 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260807 04:09:40.516850 22308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:40.519315 22291 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:09:40.519654 22291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "0c8b96d67d314cb382e9afca81f2c68f"
format_stamp: "Formatted at 2026-08-07 04:09:40 on dist-test-slave-8n49"
server_key: "38000595ed38227a4d27e4ee5b468ea1"
server_key_iv: "0822dc744620a401e99ccc8df76706d4"
server_key_version: "encryptionkey@0"
I20260807 04:09:40.520076 22291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:40.555751 22291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:40.564693 22291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:40.565145 22291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:40.595038 22291 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:41119
I20260807 04:09:40.595160 22361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:41119 every 8 connection(s)
I20260807 04:09:40.597599 22291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:40.603159 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 22291
I20260807 04:09:40.603591 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:40.604967 19204 external_mini_cluster.cc:1428] Setting key 122a2fbfc7120850670dcec4716ca48b
I20260807 04:09:40.605557 22362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:40.634136 22362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f: Bootstrap starting.
Aug 07 04:09:40 dist-test-slave-8n49 krb5kdc[22278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:40.640095 22362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:40.642398 22362 log.cc:824] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:40.647256 22362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f: No bootstrap required, opened a new log
I20260807 04:09:40.653056 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:40.608425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.18.193.62:41119)
0807 04:09:40.610116 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:40.610143 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:40.610232 (+ 89us) server_negotiation.cc:407] Connection header received
0807 04:09:40.615276 (+ 5044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:40.615318 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:40.616224 (+ 906us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:40.616993 (+ 769us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:40.620862 (+ 3869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:40.623061 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:40.626486 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:40.628630 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:40.628932 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:40.637642 (+ 8710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:40.637655 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:40.637682 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:40.637765 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:40.644426 (+ 6661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:40.645974 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:40.646012 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:40.646021 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:40.646341 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:40.647023 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:40.647029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:40.647032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:40.648082 (+ 1050us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:40.648821 (+ 739us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:40.649809 (+ 988us) server_negotiation.cc:299] Negotiation successful
0807 04:09:40.650807 (+ 998us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1283,"thread_start_us":787,"threads_started":1}
I20260807 04:09:40.660296 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:09:40.663503 22362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b96d67d314cb382e9afca81f2c68f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41119 } }
I20260807 04:09:40.664156 22362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:40.664330 22362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8b96d67d314cb382e9afca81f2c68f, State: Initialized, Role: FOLLOWER
I20260807 04:09:40.665362 22362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b96d67d314cb382e9afca81f2c68f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41119 } }
I20260807 04:09:40.665732 22362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:40.666005 22362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:40.666323 22362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:40.669919 22362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b96d67d314cb382e9afca81f2c68f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41119 } }
I20260807 04:09:40.671051 22362 leader_election.cc:302] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8b96d67d314cb382e9afca81f2c68f; no voters:
I20260807 04:09:40.672279 22362 leader_election.cc:288] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:40.672771 22367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:40.674211 22367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [term 1 LEADER]: Becoming Leader. State: Replica: 0c8b96d67d314cb382e9afca81f2c68f, State: Running, Role: LEADER
I20260807 04:09:40.675418 22367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b96d67d314cb382e9afca81f2c68f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41119 } }
I20260807 04:09:40.676009 22362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:40.681427 22370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8b96d67d314cb382e9afca81f2c68f. Latest consensus state: current_term: 1 leader_uuid: "0c8b96d67d314cb382e9afca81f2c68f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b96d67d314cb382e9afca81f2c68f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41119 } } }
I20260807 04:09:40.682247 22370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:40.682902 22368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c8b96d67d314cb382e9afca81f2c68f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b96d67d314cb382e9afca81f2c68f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41119 } } }
I20260807 04:09:40.683454 22368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:40.689613 22375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:40.708985 22375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:40.724072 22375 catalog_manager.cc:1347] Generated new cluster ID: add9a6aeb8fa4149808ec81ef41f7e69
I20260807 04:09:40.724362 22375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:40.740155 22375 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:40.743455 22375 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:40.752075 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:40.739487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.18.193.62:41119)
0807 04:09:40.739774 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:40.739783 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:40.740918 (+ 1135us) server_negotiation.cc:407] Connection header received
0807 04:09:40.741322 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:40.741335 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:40.741428 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:40.741610 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:40.743314 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:40.744725 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:40.746890 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:40.747555 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:40.747780 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:40.749647 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:40.749654 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:40.749657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:40.750751 (+ 1094us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:40.750797 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:40.750828 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:40.750958 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:40.751203 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 04:09:40.751456 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:09:40.759233 22375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c8b96d67d314cb382e9afca81f2c68f: Generated new TSK 0
I20260807 04:09:40.761142 22375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:09:40.783209 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22291
I20260807 04:09:40.785539 22357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:40.941296 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 22291
2026-08-07T04:09:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2293 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:41 dist-test-slave-8n49 krb5kdc[22397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:41 dist-test-slave-8n49 krb5kdc[22397](info): set up 2 sockets
Aug 07 04:09:41 dist-test-slave-8n49 krb5kdc[22397](info): commencing operation
krb5kdc: starting...
W20260807 04:09:42.719420 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.714s user 0.005s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:43 dist-test-slave-8n49 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:43.232265 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:42453
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:42453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:43.567328 22412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:43.567699 22412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:43.567868 22412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:43.575067 22412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:43.575217 22412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:43.575331 22412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:43.575441 22412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:43.575525 22412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:43.585383 22412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:42453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:43.586792 22412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:43.588130 22412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:43.600356 22417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:43.600432 22418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:43.600492 22420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:43.601169 22412 server_base.cc:1034] running on GCE node
I20260807 04:09:43.601902 22412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:43.603302 22412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:43.604544 22412 hybrid_clock.cc:648] HybridClock initialized: now 1786075783604504 us; error 32 us; skew 500 ppm
I20260807 04:09:43.604945 22412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:43 dist-test-slave-8n49 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075783, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:43.608208 22412 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:43.609799 22412 webserver.cc:460] Webserver started at http://127.18.193.62:39465/ using document root <none> and password file <none>
I20260807 04:09:43.610529 22412 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:43.610699 22412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:43.611013 22412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:43.614121 22412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "de5dafbefc1040b09941995c845685fc"
format_stamp: "Formatted at 2026-08-07 04:09:43 on dist-test-slave-8n49"
server_key: "99d614189cafb92862bd5ee893d02c92"
server_key_iv: "e5e5bb19da4940e4736700e4673ce230"
server_key_version: "encryptionkey@0"
I20260807 04:09:43.614912 22412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "de5dafbefc1040b09941995c845685fc"
format_stamp: "Formatted at 2026-08-07 04:09:43 on dist-test-slave-8n49"
server_key: "99d614189cafb92862bd5ee893d02c92"
server_key_iv: "e5e5bb19da4940e4736700e4673ce230"
server_key_version: "encryptionkey@0"
I20260807 04:09:43.620352 22412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:09:43.624151 22427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:43.625658 22412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:43.625857 22412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "de5dafbefc1040b09941995c845685fc"
format_stamp: "Formatted at 2026-08-07 04:09:43 on dist-test-slave-8n49"
server_key: "99d614189cafb92862bd5ee893d02c92"
server_key_iv: "e5e5bb19da4940e4736700e4673ce230"
server_key_version: "encryptionkey@0"
I20260807 04:09:43.626061 22412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:43.649567 22412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:43.654048 22412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:43.654356 22412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:43.673468 22412 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:42453
I20260807 04:09:43.673471 22480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:42453 every 8 connection(s)
I20260807 04:09:43.674899 22412 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:43.678663 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:43.683702 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 22412
I20260807 04:09:43.683938 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:43.684650 19204 external_mini_cluster.cc:1428] Setting key b3fc3e32b6859302489774c2b9fa06b8
I20260807 04:09:43.690934 22481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc: Bootstrap starting.
I20260807 04:09:43.694801 22481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:43.695818 22481 log.cc:824] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:43.699190 22481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc: No bootstrap required, opened a new log
Aug 07 04:09:43 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:43.709002 22481 raft_consensus.cc:348] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5dafbefc1040b09941995c845685fc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42453 } }
I20260807 04:09:43.709415 22481 raft_consensus.cc:374] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:43.709518 22481 raft_consensus.cc:729] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de5dafbefc1040b09941995c845685fc, State: Initialized, Role: FOLLOWER
I20260807 04:09:43.709947 22484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:43.686753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.18.193.62:42453)
0807 04:09:43.687782 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:43.687799 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:43.687850 (+ 51us) server_negotiation.cc:407] Connection header received
0807 04:09:43.690239 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:43.690267 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:43.690840 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:43.691279 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:43.692643 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:43.694609 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:43.696038 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:43.696999 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:43.697137 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:43.701924 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:43.701935 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:43.701964 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:43.702005 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:43.705382 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:43.706336 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:43.706344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:43.706350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:43.706538 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:43.706951 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:43.706954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:43.706956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:43.707551 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:43.707794 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:43.708277 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 04:09:43.708744 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":433,"threads_started":1}
I20260807 04:09:43.710170 22481 consensus_queue.cc:260] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5dafbefc1040b09941995c845685fc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42453 } }
I20260807 04:09:43.710446 22481 raft_consensus.cc:388] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:43.710603 22481 raft_consensus.cc:482] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:43.710793 22481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:43.713492 22481 raft_consensus.cc:504] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5dafbefc1040b09941995c845685fc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42453 } }
I20260807 04:09:43.714008 22481 leader_election.cc:302] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de5dafbefc1040b09941995c845685fc; no voters:
I20260807 04:09:43.714614 22481 leader_election.cc:288] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:43.714747 22486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:43.715791 22486 raft_consensus.cc:686] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [term 1 LEADER]: Becoming Leader. State: Replica: de5dafbefc1040b09941995c845685fc, State: Running, Role: LEADER
I20260807 04:09:43.716428 22486 consensus_queue.cc:237] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5dafbefc1040b09941995c845685fc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42453 } }
I20260807 04:09:43.716884 22481 sys_catalog.cc:564] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:43.719046 22488 sys_catalog.cc:455] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader de5dafbefc1040b09941995c845685fc. Latest consensus state: current_term: 1 leader_uuid: "de5dafbefc1040b09941995c845685fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5dafbefc1040b09941995c845685fc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42453 } } }
I20260807 04:09:43.719367 22488 sys_catalog.cc:458] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:43.722558 22494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:43.722402 22487 sys_catalog.cc:455] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de5dafbefc1040b09941995c845685fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5dafbefc1040b09941995c845685fc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42453 } } }
I20260807 04:09:43.722765 22487 sys_catalog.cc:458] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:43.727958 22494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:43.737110 22494 catalog_manager.cc:1347] Generated new cluster ID: 9203069e01ee478eba8c29883d415caa
I20260807 04:09:43.737238 22494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:43.756436 22494 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:43.758018 22494 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:43.767055 22494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de5dafbefc1040b09941995c845685fc: Generated new TSK 0
I20260807 04:09:43.767969 22494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:43.820917 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42453
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:44.164175 22508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:44.164506 22508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:44.164701 22508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:44.172231 22508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:44.172433 22508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:44.172638 22508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:44.182804 22508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:44.184233 22508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:44.185599 22508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:44.198884 22514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:44.198942 22515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:44.199476 22517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:44.199795 22508 server_base.cc:1034] running on GCE node
I20260807 04:09:44.200409 22508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:44.201467 22508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:44.202708 22508 hybrid_clock.cc:648] HybridClock initialized: now 1786075784202657 us; error 45 us; skew 500 ppm
I20260807 04:09:44.203118 22508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:44 dist-test-slave-8n49 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075784, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:44.206939 22508 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:44.208626 22508 webserver.cc:460] Webserver started at http://127.18.193.1:40533/ using document root <none> and password file <none>
I20260807 04:09:44.209359 22508 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:44.209513 22508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:44.209824 22508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:44.213259 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "1f5093069d22426485ebbbf37de88bee"
format_stamp: "Formatted at 2026-08-07 04:09:44 on dist-test-slave-8n49"
server_key: "2638dad2fdc390fd00814ccf2d8ff74f"
server_key_iv: "03c59d6314ef6b304cadabf94f8e0030"
server_key_version: "encryptionkey@0"
I20260807 04:09:44.214123 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "1f5093069d22426485ebbbf37de88bee"
format_stamp: "Formatted at 2026-08-07 04:09:44 on dist-test-slave-8n49"
server_key: "2638dad2fdc390fd00814ccf2d8ff74f"
server_key_iv: "03c59d6314ef6b304cadabf94f8e0030"
server_key_version: "encryptionkey@0"
I20260807 04:09:44.219702 22508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:09:44.223554 22524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:44.225033 22508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:44.225221 22508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "1f5093069d22426485ebbbf37de88bee"
format_stamp: "Formatted at 2026-08-07 04:09:44 on dist-test-slave-8n49"
server_key: "2638dad2fdc390fd00814ccf2d8ff74f"
server_key_iv: "03c59d6314ef6b304cadabf94f8e0030"
server_key_version: "encryptionkey@0"
I20260807 04:09:44.225431 22508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:44.247519 22508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:44.252229 22508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:44.252650 22508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:44.253968 22508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:44.255366 22508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:44.255473 22508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:44.255636 22508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:44.255709 22508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:44.290176 22508 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:36809
I20260807 04:09:44.290225 22641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:36809 every 8 connection(s)
I20260807 04:09:44.291538 22508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:44.294988 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 22508
I20260807 04:09:44.295220 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:44.295926 19204 external_mini_cluster.cc:1428] Setting key 0c12f0f8d7e9bad72aab66e507a5dd65
Aug 07 04:09:44 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075784, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:44.313745 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:44.294183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:42245 (local address 127.18.193.62:42453)
0807 04:09:44.294666 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:44.294675 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:44.295745 (+ 1070us) server_negotiation.cc:407] Connection header received
0807 04:09:44.297906 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:44.297911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:44.298000 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:44.298134 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:44.299610 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.300805 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.302365 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.303095 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.303240 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:44.307382 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:44.307387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:44.307390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:44.307423 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:44.310193 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:44.311212 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:44.311216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:44.311219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:44.311332 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:44.311823 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:44.311825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:44.311827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:44.312144 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:44.312293 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:44.313303 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0807 04:09:44.313479 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":199,"threads_started":1}
I20260807 04:09:44.315482 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:44.294747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:42453 (local address 127.18.193.1:42245)
0807 04:09:44.295567 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:44.295612 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:44.297587 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:44.298319 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:44.298327 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:44.298846 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:44.299428 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:44.299448 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.300957 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.300961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:44.302193 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:44.302205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.303338 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.303347 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:44.303639 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:44.304447 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:44.304484 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:44.307125 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:44.310345 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:44.310358 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:44.310379 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:44.311099 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:44.311440 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:44.311447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:44.311453 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:44.311713 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:44.312303 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:44.312315 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:44.312878 (+ 563us) client_negotiation.cc:769] Sending connection context
0807 04:09:44.313220 (+ 342us) client_negotiation.cc:240] Negotiation successful
0807 04:09:44.313859 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":345,"threads_started":1}
I20260807 04:09:44.317306 22642 heartbeater.cc:347] Connected to a master server at 127.18.193.62:42453
I20260807 04:09:44.317703 22642 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:44.318565 22642 heartbeater.cc:511] Master 127.18.193.62:42453 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
I20260807 04:09:44.321053 22444 ts_manager.cc:194] Registered new tserver with Master: 1f5093069d22426485ebbbf37de88bee (127.18.193.1:36809)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
I20260807 04:09:44.322844 22444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:42245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:44.353730 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42453
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:44.692766 22649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:44.693141 22649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:44.693323 22649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:44.700882 22649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:44.701063 22649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:44.701256 22649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:44.711901 22649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:44.713519 22649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:44.714886 22649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:44.729121 22654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:44.729168 22657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:44.729204 22655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:44.730064 22649 server_base.cc:1034] running on GCE node
I20260807 04:09:44.730700 22649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:44.731745 22649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:44.732952 22649 hybrid_clock.cc:648] HybridClock initialized: now 1786075784732888 us; error 60 us; skew 500 ppm
I20260807 04:09:44.733295 22649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:44 dist-test-slave-8n49 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075784, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:44.736694 22649 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:44.738279 22649 webserver.cc:460] Webserver started at http://127.18.193.2:36445/ using document root <none> and password file <none>
I20260807 04:09:44.739008 22649 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:44.739142 22649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:44.739457 22649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:44.742465 22649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "4eba4fe11ca84c95a520c0478dccbbff"
format_stamp: "Formatted at 2026-08-07 04:09:44 on dist-test-slave-8n49"
server_key: "68b10989e5b419f94e8b02838fa6e04d"
server_key_iv: "56a081884b158328efe7bd8128730710"
server_key_version: "encryptionkey@0"
I20260807 04:09:44.743294 22649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "4eba4fe11ca84c95a520c0478dccbbff"
format_stamp: "Formatted at 2026-08-07 04:09:44 on dist-test-slave-8n49"
server_key: "68b10989e5b419f94e8b02838fa6e04d"
server_key_iv: "56a081884b158328efe7bd8128730710"
server_key_version: "encryptionkey@0"
I20260807 04:09:44.748481 22649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:09:44.751922 22664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:44.753273 22649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:44.753439 22649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "4eba4fe11ca84c95a520c0478dccbbff"
format_stamp: "Formatted at 2026-08-07 04:09:44 on dist-test-slave-8n49"
server_key: "68b10989e5b419f94e8b02838fa6e04d"
server_key_iv: "56a081884b158328efe7bd8128730710"
server_key_version: "encryptionkey@0"
I20260807 04:09:44.753634 22649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:44.776340 22649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:44.780830 22649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:44.781147 22649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:44.782150 22649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:44.783541 22649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:44.783622 22649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:44.783776 22649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:44.783828 22649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:44.818024 22649 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:33221
I20260807 04:09:44.818166 22780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:33221 every 8 connection(s)
I20260807 04:09:44.819393 22649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:44.819944 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 22649
I20260807 04:09:44.820163 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:44.820798 19204 external_mini_cluster.cc:1428] Setting key 429b23a3cf9e33d364a128a9a58cca67
Aug 07 04:09:44 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075784, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:44.842870 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:44.822188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:45983 (local address 127.18.193.62:42453)
0807 04:09:44.822666 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:44.822672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:44.823774 (+ 1102us) server_negotiation.cc:407] Connection header received
0807 04:09:44.825733 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:44.825742 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:44.825847 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:44.825978 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:44.827689 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.828930 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.830684 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.831443 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.831590 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:44.835333 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:44.835341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:44.835350 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:44.835414 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:44.839163 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:44.839965 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:44.839973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:44.839979 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:44.840152 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:44.840701 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:44.840709 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:44.840715 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:44.841162 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:44.841391 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:44.842149 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 04:09:44.842416 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":235,"threads_started":1}
I20260807 04:09:44.843607 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:44.822786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:42453 (local address 127.18.193.2:45983)
0807 04:09:44.823599 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:44.823652 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:44.825453 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:44.826215 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:44.826225 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:44.826827 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:44.827477 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:44.827497 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.829102 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.829110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:44.830533 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:44.830547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:44.831667 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:44.831672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:44.831875 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:44.832633 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:44.832657 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:44.835083 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:44.839338 (+ 4255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:44.839346 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:44.839361 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:44.839818 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:44.840302 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:44.840307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:44.840309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:44.840519 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:44.841330 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:44.841337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:44.841718 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 04:09:44.841994 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 04:09:44.842412 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":348,"threads_started":1}
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
I20260807 04:09:44.845360 22781 heartbeater.cc:347] Connected to a master server at 127.18.193.62:42453
I20260807 04:09:44.845741 22781 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:44.846603 22781 heartbeater.cc:511] Master 127.18.193.62:42453 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
I20260807 04:09:44.848510 22444 ts_manager.cc:194] Registered new tserver with Master: 4eba4fe11ca84c95a520c0478dccbbff (127.18.193.2:33221)
I20260807 04:09:44.849440 22444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:45983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:44.877254 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42453
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:45.216372 22787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:45.216739 22787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:45.216904 22787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:45.224213 22787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:45.224318 22787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:45.224490 22787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:45.234915 22787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:45.236214 22787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:45.237643 22787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:45.250759 22793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:45.250837 22794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:45.250792 22796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:45.251883 22787 server_base.cc:1034] running on GCE node
I20260807 04:09:45.252530 22787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:45.253635 22787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:45.254835 22787 hybrid_clock.cc:648] HybridClock initialized: now 1786075785254777 us; error 57 us; skew 500 ppm
I20260807 04:09:45.255149 22787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:45.258832 22787 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:45.260314 22787 webserver.cc:460] Webserver started at http://127.18.193.3:41015/ using document root <none> and password file <none>
I20260807 04:09:45.261178 22787 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:45.261322 22787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:45.261641 22787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:45.264701 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "1921e00536c94191be56e52f0d27fbb7"
format_stamp: "Formatted at 2026-08-07 04:09:45 on dist-test-slave-8n49"
server_key: "d4eed8902c1f955dcbcac33556d78d30"
server_key_iv: "02e0ac9d4bb9f43fbbecd84ac5c4996a"
server_key_version: "encryptionkey@0"
I20260807 04:09:45.265547 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "1921e00536c94191be56e52f0d27fbb7"
format_stamp: "Formatted at 2026-08-07 04:09:45 on dist-test-slave-8n49"
server_key: "d4eed8902c1f955dcbcac33556d78d30"
server_key_iv: "02e0ac9d4bb9f43fbbecd84ac5c4996a"
server_key_version: "encryptionkey@0"
I20260807 04:09:45.271044 22787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:09:45.274832 22803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:45.276245 22787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:09:45.276435 22787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "1921e00536c94191be56e52f0d27fbb7"
format_stamp: "Formatted at 2026-08-07 04:09:45 on dist-test-slave-8n49"
server_key: "d4eed8902c1f955dcbcac33556d78d30"
server_key_iv: "02e0ac9d4bb9f43fbbecd84ac5c4996a"
server_key_version: "encryptionkey@0"
I20260807 04:09:45.276700 22787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:45.294518 22787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:45.299134 22787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:45.299463 22787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:45.300526 22787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:45.302006 22787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:45.302109 22787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:45.302277 22787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:45.302347 22787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:45.326776 22642 heartbeater.cc:503] Master 127.18.193.62:42453 was elected leader, sending a full tablet report...
I20260807 04:09:45.339769 22787 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:34121
I20260807 04:09:45.339819 22919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:34121 every 8 connection(s)
I20260807 04:09:45.341171 22787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:45.349119 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 22787
I20260807 04:09:45.349304 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:45.349848 19204 external_mini_cluster.cc:1428] Setting key fec4f2ba0635bf77e1e0e91f7cfda71a
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:45.361912 22922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:45.343692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:53411 (local address 127.18.193.62:42453)
0807 04:09:45.344155 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:45.344163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:45.345236 (+ 1073us) server_negotiation.cc:407] Connection header received
0807 04:09:45.347006 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:45.347010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:45.347100 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:45.347199 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:45.348702 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.349628 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.351451 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.351976 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.352117 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:45.355625 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:45.355631 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:45.355634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:45.355671 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:45.358736 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.359471 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.359477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.359484 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.359605 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.360043 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.360048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.360052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.360334 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:45.360501 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:45.361269 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 04:09:45.361493 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":202,"threads_started":1}
I20260807 04:09:45.362902 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:45.344320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:42453 (local address 127.18.193.3:53411)
0807 04:09:45.345063 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:45.345116 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:45.346733 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:45.347407 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:45.347415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:45.347908 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:45.348481 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:45.348499 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.349782 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.349787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:45.351310 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:45.351326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.352184 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.352189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:45.352398 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:45.353086 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:45.353118 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:45.355387 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:45.358877 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:45.358887 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:45.358905 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:45.359340 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:45.359715 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:45.359720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:45.359723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:45.359921 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:45.360477 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:45.360486 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:45.360900 (+ 414us) client_negotiation.cc:769] Sending connection context
0807 04:09:45.361161 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 04:09:45.361724 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":309,"threads_started":1}
I20260807 04:09:45.364344 22920 heartbeater.cc:347] Connected to a master server at 127.18.193.62:42453
I20260807 04:09:45.364713 22920 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:45.365686 22920 heartbeater.cc:511] Master 127.18.193.62:42453 requested a full tablet report, sending...
I20260807 04:09:45.367043 22444 ts_manager.cc:194] Registered new tserver with Master: 1921e00536c94191be56e52f0d27fbb7 (127.18.193.3:34121)
I20260807 04:09:45.367805 22444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:53411
I20260807 04:09:45.378238 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075785, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075785, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:45.433362 22922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:45.418055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.18.193.62:42453)
0807 04:09:45.418322 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:45.418331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:45.418732 (+ 401us) server_negotiation.cc:407] Connection header received
0807 04:09:45.418943 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:45.418948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:45.419021 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:45.419103 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:45.420689 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.421432 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.422751 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.423155 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.423254 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:45.427457 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:45.427466 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:45.427475 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:45.427524 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:45.430723 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.431402 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.431406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.431409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.431531 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.431971 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.431975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.431977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.432334 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:45.432621 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:45.432865 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 04:09:45.433041 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 04:09:45.434084 22444 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59270
I20260807 04:09:45.436524 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22508
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075785, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:45.453594 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:45.437695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.18.193.1:36809)
0807 04:09:45.438400 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:45.438407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:45.438435 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:09:45.438493 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:45.438499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:45.438704 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:45.438834 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:45.440100 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.441226 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.442531 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.443111 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.443221 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:45.447492 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:45.447506 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:45.447521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:45.447558 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:45.450566 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.451233 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.451245 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.451247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.451373 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.451806 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.451811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.451813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.452231 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:45.452534 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:45.452941 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 04:09:45.453170 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":396,"threads_started":1}
W20260807 04:09:45.454910 22637 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59366
W20260807 04:09:45.456117 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22508: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:45.456382 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 22508
I20260807 04:09:45.474677 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22649
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075785, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:09:45.494975 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:45.475606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.18.193.2:33221)
0807 04:09:45.476152 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:45.476159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:45.476186 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:09:45.476259 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:45.476265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:45.476475 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:45.476679 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:45.477922 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.478949 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.480367 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.481202 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.481341 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:45.485890 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:45.485903 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:45.485919 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:45.485957 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:45.492225 (+ 6268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.493108 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.493115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.493117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.493252 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.493613 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.493616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.493618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.493963 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:45.494191 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:45.494471 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 04:09:45.494681 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":291,"threads_started":1}
W20260807 04:09:45.496330 22776 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57308
W20260807 04:09:45.497493 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:45.497660 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 22649
I20260807 04:09:45.516634 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22787
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075785, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:09:45.535303 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:45.517914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.18.193.3:34121)
0807 04:09:45.518763 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:45.518770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:45.518801 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:09:45.518869 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:45.518876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:45.519093 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:45.519230 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:45.520801 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.521871 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.523227 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:45.524071 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:45.524187 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:45.529282 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:45.529295 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:45.529311 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:45.529351 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:45.532577 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.533232 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.533238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.533241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.533355 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:45.533712 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:45.533715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:45.533717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:45.534075 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:45.534305 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:45.534665 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 04:09:45.534870 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":509,"threads_started":1}
W20260807 04:09:45.536887 22915 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54920
W20260807 04:09:45.538149 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22787: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:09:45.538343 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 22787
I20260807 04:09:45.556782 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22412
I20260807 04:09:45.558043 22476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:45.664811 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 22412
2026-08-07T04:09:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4708 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22939](info): set up 2 sockets
Aug 07 04:09:45 dist-test-slave-8n49 krb5kdc[22939](info): commencing operation
krb5kdc: starting...
W20260807 04:09:46.811233 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:46 dist-test-slave-8n49 krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:47.049423 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33917
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:33917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:47.396695 22953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:47.396998 22953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:47.397099 22953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:47.404594 22953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:47.404716 22953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:47.404763 22953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:47.404821 22953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:47.404883 22953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:47.415513 22953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:33917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:47.416911 22953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:47.418196 22953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:47.430527 22960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:47.430498 22959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:47.430616 22962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:47.431443 22953 server_base.cc:1034] running on GCE node
I20260807 04:09:47.432318 22953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:47.433945 22953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:47.435170 22953 hybrid_clock.cc:648] HybridClock initialized: now 1786075787435103 us; error 61 us; skew 500 ppm
I20260807 04:09:47.435544 22953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:47 dist-test-slave-8n49 krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075787, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:47.438766 22953 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:47.440265 22953 webserver.cc:460] Webserver started at http://127.18.193.62:45391/ using document root <none> and password file <none>
I20260807 04:09:47.441033 22953 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:47.441138 22953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:47.441398 22953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:47.444415 22953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "0eb1945692ac4ee4ab73bfb85e2da545"
format_stamp: "Formatted at 2026-08-07 04:09:47 on dist-test-slave-8n49"
server_key: "403962aade50c98b2c1eefb54fd6fb0a"
server_key_iv: "306eff34f2258b49f2457ee33bfeaf50"
server_key_version: "encryptionkey@0"
I20260807 04:09:47.445261 22953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "0eb1945692ac4ee4ab73bfb85e2da545"
format_stamp: "Formatted at 2026-08-07 04:09:47 on dist-test-slave-8n49"
server_key: "403962aade50c98b2c1eefb54fd6fb0a"
server_key_iv: "306eff34f2258b49f2457ee33bfeaf50"
server_key_version: "encryptionkey@0"
I20260807 04:09:47.450613 22953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:09:47.454202 22969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:47.455569 22953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:09:47.455751 22953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "0eb1945692ac4ee4ab73bfb85e2da545"
format_stamp: "Formatted at 2026-08-07 04:09:47 on dist-test-slave-8n49"
server_key: "403962aade50c98b2c1eefb54fd6fb0a"
server_key_iv: "306eff34f2258b49f2457ee33bfeaf50"
server_key_version: "encryptionkey@0"
I20260807 04:09:47.455986 22953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:47.479172 22953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:47.483266 22953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:47.483569 22953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:47.502835 22953 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:33917
I20260807 04:09:47.502837 23022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:33917 every 8 connection(s)
I20260807 04:09:47.504256 22953 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:47.508095 23023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:47.513266 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 22953
I20260807 04:09:47.513506 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:47.514156 19204 external_mini_cluster.cc:1428] Setting key 6a134880f47ae3a10634c59f65fcd120
I20260807 04:09:47.519323 23023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545: Bootstrap starting.
I20260807 04:09:47.522537 23023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:47.523627 23023 log.cc:824] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:47.526597 23023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545: No bootstrap required, opened a new log
Aug 07 04:09:47 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:47.535218 23023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33917 } }
I20260807 04:09:47.535622 23023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:47.535744 23023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eb1945692ac4ee4ab73bfb85e2da545, State: Initialized, Role: FOLLOWER
I20260807 04:09:47.536365 23023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33917 } }
I20260807 04:09:47.536705 23023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:47.536871 23023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:47.537071 23023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:47.538885 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:47.516241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.18.193.62:33917)
0807 04:09:47.517153 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:47.517169 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:47.517219 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:09:47.519148 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:47.519173 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:47.519720 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:47.520158 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:47.521877 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:47.523417 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:47.525377 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:47.526116 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:47.526337 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:47.531001 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:47.531013 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:47.531028 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:47.531069 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:47.534435 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:47.535178 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:47.535185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:47.535191 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:47.535365 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:47.535875 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:47.535880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:47.535885 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:47.536531 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:47.536902 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:47.537371 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 04:09:47.537774 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":427,"threads_started":1}
I20260807 04:09:47.539777 23023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33917 } }
I20260807 04:09:47.540302 23023 leader_election.cc:302] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0eb1945692ac4ee4ab73bfb85e2da545; no voters:
I20260807 04:09:47.541064 23023 leader_election.cc:288] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:47.541184 23028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:47.543764 23023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:47.548270 23028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [term 1 LEADER]: Becoming Leader. State: Replica: 0eb1945692ac4ee4ab73bfb85e2da545, State: Running, Role: LEADER
I20260807 04:09:47.549172 23028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33917 } }
I20260807 04:09:47.552538 23029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33917 } } }
I20260807 04:09:47.553037 23029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:47.553205 23030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eb1945692ac4ee4ab73bfb85e2da545. Latest consensus state: current_term: 1 leader_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb1945692ac4ee4ab73bfb85e2da545" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33917 } } }
I20260807 04:09:47.553556 23030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:47.554744 23043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:47.558916 23043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:47.567190 23043 catalog_manager.cc:1347] Generated new cluster ID: 7a15eb78c45c4edc96db4502b1c074db
I20260807 04:09:47.567351 23043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:47.586498 23043 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:47.588495 23043 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:47.598909 23043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0eb1945692ac4ee4ab73bfb85e2da545: Generated new TSK 0
I20260807 04:09:47.599834 23043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:47.668962 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33917
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:48.021304 23049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:48.021811 23049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:48.022071 23049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:48.030068 23049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:48.030232 23049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:48.030434 23049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:48.040910 23049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:48.042258 23049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:48.043625 23049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:48.058204 23054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:48.058172 23058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:48.058204 23055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:48.059459 23049 server_base.cc:1034] running on GCE node
I20260807 04:09:48.060127 23049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:48.061198 23049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:48.062438 23049 hybrid_clock.cc:648] HybridClock initialized: now 1786075788062388 us; error 39 us; skew 500 ppm
I20260807 04:09:48.062779 23049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:48 dist-test-slave-8n49 krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:48.066383 23049 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:48.067956 23049 webserver.cc:460] Webserver started at http://127.18.193.1:42909/ using document root <none> and password file <none>
I20260807 04:09:48.068832 23049 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:48.069002 23049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:48.069339 23049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:48.072340 23049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "51850d68fc70444cb70cf79d264a1cbd"
format_stamp: "Formatted at 2026-08-07 04:09:48 on dist-test-slave-8n49"
server_key: "1b2c84faaadfb92b6d6ed549997e017d"
server_key_iv: "14b043ccecabb373249e18ab845c6c35"
server_key_version: "encryptionkey@0"
I20260807 04:09:48.073262 23049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "51850d68fc70444cb70cf79d264a1cbd"
format_stamp: "Formatted at 2026-08-07 04:09:48 on dist-test-slave-8n49"
server_key: "1b2c84faaadfb92b6d6ed549997e017d"
server_key_iv: "14b043ccecabb373249e18ab845c6c35"
server_key_version: "encryptionkey@0"
I20260807 04:09:48.078681 23049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 04:09:48.082509 23066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:48.083984 23049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:09:48.084187 23049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "51850d68fc70444cb70cf79d264a1cbd"
format_stamp: "Formatted at 2026-08-07 04:09:48 on dist-test-slave-8n49"
server_key: "1b2c84faaadfb92b6d6ed549997e017d"
server_key_iv: "14b043ccecabb373249e18ab845c6c35"
server_key_version: "encryptionkey@0"
I20260807 04:09:48.084429 23049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:48.097432 23049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:48.102082 23049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:48.102402 23049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:48.103430 23049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:48.105664 23049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:48.105787 23049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:48.105939 23049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:48.106014 23049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:48.140216 23049 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:36995
I20260807 04:09:48.140259 23181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:36995 every 8 connection(s)
I20260807 04:09:48.141633 23049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:48.143082 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 23049
I20260807 04:09:48.143316 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:48.144129 19204 external_mini_cluster.cc:1428] Setting key 3106aed080f593014744ff63b3542b57
Aug 07 04:09:48 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:48.165145 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:48.144645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:55787 (local address 127.18.193.62:33917)
0807 04:09:48.145220 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:48.145226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:48.146179 (+ 953us) server_negotiation.cc:407] Connection header received
0807 04:09:48.148192 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:48.148198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:48.148287 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:48.148411 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:48.149962 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.150722 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.152271 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.152779 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.152915 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:48.157223 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:48.157233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:48.157240 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:48.157289 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:48.161191 (+ 3902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:48.162045 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:48.162054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:48.162061 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:48.162239 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:48.162776 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:48.162784 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:48.162791 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:48.163208 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:48.163371 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:48.164288 (+ 917us) server_negotiation.cc:299] Negotiation successful
0807 04:09:48.164615 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":278,"threads_started":1}
I20260807 04:09:48.165827 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:48.145264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33917 (local address 127.18.193.1:55787)
0807 04:09:48.146027 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:48.146074 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:48.147898 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:48.148682 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:48.148690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:48.149195 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:48.149788 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:48.149806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.150878 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.150882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:48.152129 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:48.152141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.153023 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.153032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:48.153273 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:48.154020 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:48.154041 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:48.156959 (+ 2918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:48.161369 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:48.161377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:48.161393 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:48.161889 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:48.162387 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:48.162391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:48.162394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:48.162629 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:48.163336 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:48.163343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:48.163857 (+ 514us) client_negotiation.cc:769] Sending connection context
0807 04:09:48.164110 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 04:09:48.164597 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":294,"threads_started":1}
I20260807 04:09:48.167488 23182 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33917
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
I20260807 04:09:48.167887 23182 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:48.168730 23182 heartbeater.cc:511] Master 127.18.193.62:33917 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
I20260807 04:09:48.171068 22987 ts_manager.cc:194] Registered new tserver with Master: 51850d68fc70444cb70cf79d264a1cbd (127.18.193.1:36995)
I20260807 04:09:48.172538 22987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:55787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:48.201052 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33917
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:48.553937 23188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:48.554250 23188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:48.554425 23188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:48.562247 23188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:48.562408 23188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:48.562626 23188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:48.572868 23188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:48.574265 23188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:48.575666 23188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:48.589432 23193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:48.589514 23194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:48.589473 23196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:48.591094 23188 server_base.cc:1034] running on GCE node
I20260807 04:09:48.591832 23188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:48.592990 23188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:48.594236 23188 hybrid_clock.cc:648] HybridClock initialized: now 1786075788594161 us; error 65 us; skew 500 ppm
I20260807 04:09:48.594604 23188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:48 dist-test-slave-8n49 krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:48.598007 23188 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:48.599689 23188 webserver.cc:460] Webserver started at http://127.18.193.2:43787/ using document root <none> and password file <none>
I20260807 04:09:48.600464 23188 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:48.600649 23188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:48.601004 23188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:48.604351 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "c806e31b4c8349f1a86fdde61c7f1028"
format_stamp: "Formatted at 2026-08-07 04:09:48 on dist-test-slave-8n49"
server_key: "487a3e682623827e2b6212e8ec2963e1"
server_key_iv: "df12a979b0d538ec09f3a197a53ddc09"
server_key_version: "encryptionkey@0"
I20260807 04:09:48.605319 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "c806e31b4c8349f1a86fdde61c7f1028"
format_stamp: "Formatted at 2026-08-07 04:09:48 on dist-test-slave-8n49"
server_key: "487a3e682623827e2b6212e8ec2963e1"
server_key_iv: "df12a979b0d538ec09f3a197a53ddc09"
server_key_version: "encryptionkey@0"
I20260807 04:09:48.611233 23188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:09:48.616215 23204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:48.617915 23188 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:09:48.618158 23188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "c806e31b4c8349f1a86fdde61c7f1028"
format_stamp: "Formatted at 2026-08-07 04:09:48 on dist-test-slave-8n49"
server_key: "487a3e682623827e2b6212e8ec2963e1"
server_key_iv: "df12a979b0d538ec09f3a197a53ddc09"
server_key_version: "encryptionkey@0"
I20260807 04:09:48.618412 23188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:48.642019 23188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:48.647848 23188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:48.648216 23188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:48.649478 23188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:48.651290 23188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:48.651422 23188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:48.651593 23188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:48.651666 23188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:48.690310 23188 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:43163
I20260807 04:09:48.690359 23324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:43163 every 8 connection(s)
I20260807 04:09:48.691728 23188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:48.695293 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 23188
I20260807 04:09:48.695506 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:48.695967 19204 external_mini_cluster.cc:1428] Setting key 625014420c09a854014838c2c60349cb
Aug 07 04:09:48 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:48.714815 23327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:48.694578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:43999 (local address 127.18.193.62:33917)
0807 04:09:48.695087 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:48.695094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:48.696291 (+ 1197us) server_negotiation.cc:407] Connection header received
0807 04:09:48.698556 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:48.698562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:48.698655 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:48.698781 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:48.700425 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.701601 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.703465 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.704271 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.704439 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:48.708407 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:48.708412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:48.708415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:48.708453 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:48.711227 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:48.712119 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:48.712126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:48.712129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:48.712274 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:48.712791 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:48.712797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:48.712800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:48.713102 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:48.713269 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:48.714154 (+ 885us) server_negotiation.cc:299] Negotiation successful
0807 04:09:48.714402 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":221,"threads_started":1}
I20260807 04:09:48.715858 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:48.695192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33917 (local address 127.18.193.2:43999)
0807 04:09:48.696105 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:48.696154 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:48.698265 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:48.699046 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:48.699056 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:48.699604 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:48.700225 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:48.700244 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.701790 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.701794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:48.703227 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:48.703243 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:48.704613 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:48.704624 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:48.704875 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:48.705799 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:48.705828 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:48.708103 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:48.711405 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:48.711416 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:48.711437 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:48.711985 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:48.712403 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:48.712409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:48.712413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:48.712677 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:48.713250 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:48.713259 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:48.713709 (+ 450us) client_negotiation.cc:769] Sending connection context
0807 04:09:48.714040 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 04:09:48.714628 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":403,"threads_started":1}
I20260807 04:09:48.717780 23325 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33917
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
I20260807 04:09:48.718122 23325 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:48.718953 23325 heartbeater.cc:511] Master 127.18.193.62:33917 requested a full tablet report, sending...
I20260807 04:09:48.720785 22987 ts_manager.cc:194] Registered new tserver with Master: c806e31b4c8349f1a86fdde61c7f1028 (127.18.193.2:43163)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
I20260807 04:09:48.721709 22987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:43999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:48.753810 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33917
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:38169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:49.107910 23332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:49.108263 23332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:49.108443 23332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:49.116088 23332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:49.116264 23332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:49.116468 23332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:49.127017 23332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:49.128422 23332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:49.129854 23332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:49.143536 23337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:49.143805 23340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:49.144781 23338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:49.145015 23332 server_base.cc:1034] running on GCE node
I20260807 04:09:49.145869 23332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:49.147224 23332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:49.148514 23332 hybrid_clock.cc:648] HybridClock initialized: now 1786075789148425 us; error 80 us; skew 500 ppm
I20260807 04:09:49.148910 23332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075789, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:49.152231 23332 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:49.153751 23332 webserver.cc:460] Webserver started at http://127.18.193.3:33867/ using document root <none> and password file <none>
I20260807 04:09:49.154460 23332 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:49.154567 23332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:49.154871 23332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:49.157991 23332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "815e53c5e3bb4620a03845fa12affbf2"
format_stamp: "Formatted at 2026-08-07 04:09:49 on dist-test-slave-8n49"
server_key: "91b7a19f0abfa2d3f02f54991b6a02e3"
server_key_iv: "89c29f567c73c37c9b8a71086e6f802f"
server_key_version: "encryptionkey@0"
I20260807 04:09:49.158762 23332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "815e53c5e3bb4620a03845fa12affbf2"
format_stamp: "Formatted at 2026-08-07 04:09:49 on dist-test-slave-8n49"
server_key: "91b7a19f0abfa2d3f02f54991b6a02e3"
server_key_iv: "89c29f567c73c37c9b8a71086e6f802f"
server_key_version: "encryptionkey@0"
I20260807 04:09:49.164397 23332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:09:49.168169 23347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:49.169674 23332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:49.169819 23332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "815e53c5e3bb4620a03845fa12affbf2"
format_stamp: "Formatted at 2026-08-07 04:09:49 on dist-test-slave-8n49"
server_key: "91b7a19f0abfa2d3f02f54991b6a02e3"
server_key_iv: "89c29f567c73c37c9b8a71086e6f802f"
server_key_version: "encryptionkey@0"
I20260807 04:09:49.170011 23332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:49.176443 23182 heartbeater.cc:503] Master 127.18.193.62:33917 was elected leader, sending a full tablet report...
I20260807 04:09:49.184092 23332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:49.188896 23332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:49.189149 23332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:49.190148 23332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:49.191516 23332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:49.191592 23332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:49.191674 23332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:49.191723 23332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:49.227923 23332 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:43689
I20260807 04:09:49.227952 23463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:43689 every 8 connection(s)
I20260807 04:09:49.229482 23332 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:49.238660 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 23332
I20260807 04:09:49.238911 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:49.239511 19204 external_mini_cluster.cc:1428] Setting key bb9d8bb5209588f9da057eb3314028c9
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075789, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:49.252440 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:49.232314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:38721 (local address 127.18.193.62:33917)
0807 04:09:49.232868 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:49.232873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:49.233864 (+ 991us) server_negotiation.cc:407] Connection header received
0807 04:09:49.235853 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:49.235859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:49.235975 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:49.236103 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:49.237866 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.238906 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.240921 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.241363 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.241517 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:49.245798 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:49.245803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:49.245806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:49.245842 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:49.248960 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.249822 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.249826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.249829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.249961 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.250442 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.250447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.250449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.250734 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:49.250907 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:49.251690 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 04:09:49.251940 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":342,"threads_started":1}
I20260807 04:09:49.253505 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:49.232865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33917 (local address 127.18.193.3:38721)
0807 04:09:49.233682 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:49.233731 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:49.235539 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:49.236373 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:49.236382 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:49.237031 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:49.237670 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:49.237689 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.239106 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.239111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:49.240758 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:49.240773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.241635 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.241645 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:49.241908 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:49.242659 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:49.242683 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:49.245526 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:49.249136 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:49.249153 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:49.249173 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:49.249663 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:49.250077 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:49.250085 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:49.250090 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:49.250306 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:49.250883 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:49.250892 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:49.251281 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 04:09:49.251568 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 04:09:49.252188 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":302,"threads_started":1}
I20260807 04:09:49.255474 23464 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33917
I20260807 04:09:49.255878 23464 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:49.256735 23464 heartbeater.cc:511] Master 127.18.193.62:33917 requested a full tablet report, sending...
I20260807 04:09:49.258147 22987 ts_manager.cc:194] Registered new tserver with Master: 815e53c5e3bb4620a03845fa12affbf2 (127.18.193.3:43689)
I20260807 04:09:49.258978 22987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:38721
I20260807 04:09:49.268297 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:49.326045 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:49.308183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.18.193.1:36995)
0807 04:09:49.308874 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:49.308882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:49.308907 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:09:49.308970 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:49.308977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:49.309197 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:49.309381 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:49.310939 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.311983 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.313713 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.314318 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.314470 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:49.318760 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:49.318774 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:49.318789 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:49.318833 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:49.322670 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.323538 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.323549 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.323556 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.323756 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.324235 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.324239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.324241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.324812 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:49.325065 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:49.325385 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 04:09:49.325674 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":351,"threads_started":1}
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:49.345152 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:49.330814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.18.193.62:33917)
0807 04:09:49.331037 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:49.331043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:49.331201 (+ 158us) server_negotiation.cc:407] Connection header received
0807 04:09:49.331418 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:49.331424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:49.331496 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:49.331601 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:49.332936 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.333666 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.334919 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.335333 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.335464 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:49.339480 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:49.339486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:49.339489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:49.339559 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:49.342525 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.343200 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.343204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.343207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.343343 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.343774 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.343778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.343780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.344123 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:49.344335 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:49.344709 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 04:09:49.344909 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 04:09:49.345817 22987 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42276
I20260807 04:09:49.346992 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 23049
I20260807 04:09:49.360729 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:49.347947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.18.193.1:36995)
0807 04:09:49.348198 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:49.348209 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:49.348508 (+ 299us) server_negotiation.cc:407] Connection header received
0807 04:09:49.348730 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:49.348741 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:49.348809 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:49.348928 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:49.350254 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.351003 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.352348 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.352888 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.353042 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:49.355471 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:49.355477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:49.355480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:49.355515 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:49.358110 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.358741 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.358748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.358752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.358882 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.359270 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.359276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.359280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.359538 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:49.359792 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:49.360143 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 04:09:49.360375 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:09:49.361343 23176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:49.449960 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 23049
I20260807 04:09:49.469398 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 23188
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:09:49.488381 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:49.470495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56308 (local address 127.18.193.2:43163)
0807 04:09:49.471133 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:49.471140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:49.471172 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:09:49.471243 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:49.471274 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:49.471491 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:49.471682 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:49.473146 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.474893 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.476284 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.476996 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.477109 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:49.481584 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:49.481597 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:49.481614 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:49.481663 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:49.485286 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.486080 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.486088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.486091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.486253 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.486706 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.486709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.486711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.487086 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:49.487320 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:49.487683 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 04:09:49.487915 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":340,"threads_started":1}
I20260807 04:09:49.490603 23319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:49.578128 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 23188
I20260807 04:09:49.597822 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 23332
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:09:49.615960 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:49.598882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.18.193.3:43689)
0807 04:09:49.599609 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:49.599617 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:49.599647 (+ 30us) server_negotiation.cc:407] Connection header received
0807 04:09:49.599714 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:49.599720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:49.599965 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:49.600114 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:49.601591 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.602666 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.604167 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:49.604905 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:49.605020 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:49.609641 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:49.609656 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:49.609672 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:49.609713 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:49.613041 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.613803 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.613810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.613812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.613944 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:49.614333 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:49.614337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:49.614340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:49.614719 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:49.614977 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:49.615291 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 04:09:49.615513 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":396,"threads_started":1}
I20260807 04:09:49.617512 23459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:49.708361 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 23332
I20260807 04:09:49.726387 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 22953
I20260807 04:09:49.727763 23018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:49.837891 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 22953
2026-08-07T04:09:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4174 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[23486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[23486](info): set up 2 sockets
Aug 07 04:09:49 dist-test-slave-8n49 krb5kdc[23486](info): commencing operation
krb5kdc: starting...
W20260807 04:09:51.007073 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:09:51 dist-test-slave-8n49 krb5kdc[23486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:09:51.257633 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35813
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:35813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:51.617967 23500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:51.618345 23500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:51.618552 23500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:51.627094 23500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:51.627240 23500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:51.627346 23500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:51.627461 23500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:51.627548 23500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:51.639322 23500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:35813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:51.640933 23500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:51.642482 23500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:51.656100 23506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:51.656119 23505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:51.656147 23508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:51.657037 23500 server_base.cc:1034] running on GCE node
I20260807 04:09:51.658000 23500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:51.659745 23500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:51.661046 23500 hybrid_clock.cc:648] HybridClock initialized: now 1786075791661006 us; error 69 us; skew 500 ppm
I20260807 04:09:51.661505 23500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:51 dist-test-slave-8n49 krb5kdc[23486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075791, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:51.671589 23500 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:09:51.673141 23500 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:51.673285 23500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:51.673673 23500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:51.677028 23500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a"
format_stamp: "Formatted at 2026-08-07 04:09:51 on dist-test-slave-8n49"
server_key: "1415c253c980aaafe6c8f47e1c4ec96d"
server_key_iv: "1db0f542aa733989cf9f8c9c8a7b4a13"
server_key_version: "encryptionkey@0"
I20260807 04:09:51.677912 23500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a"
format_stamp: "Formatted at 2026-08-07 04:09:51 on dist-test-slave-8n49"
server_key: "1415c253c980aaafe6c8f47e1c4ec96d"
server_key_iv: "1db0f542aa733989cf9f8c9c8a7b4a13"
server_key_version: "encryptionkey@0"
I20260807 04:09:51.683588 23500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:09:51.687770 23514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:51.689363 23500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:51.689596 23500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a"
format_stamp: "Formatted at 2026-08-07 04:09:51 on dist-test-slave-8n49"
server_key: "1415c253c980aaafe6c8f47e1c4ec96d"
server_key_iv: "1db0f542aa733989cf9f8c9c8a7b4a13"
server_key_version: "encryptionkey@0"
I20260807 04:09:51.689857 23500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:51.707541 23500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:51.712237 23500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:51.712589 23500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:51.732614 23500 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:35813
I20260807 04:09:51.732653 23568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:35813 every 8 connection(s)
I20260807 04:09:51.733649 23500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:51.737534 23569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:51.743639 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 23500
I20260807 04:09:51.743911 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:51.744519 19204 external_mini_cluster.cc:1428] Setting key 3e3fe879e3aa8085cce2de543664e347
I20260807 04:09:51.748726 23569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a: Bootstrap starting.
I20260807 04:09:51.752415 23569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:51.753718 23569 log.cc:824] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:51.757404 23569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a: No bootstrap required, opened a new log
Aug 07 04:09:51 dist-test-slave-8n49 krb5kdc[23486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:51.769273 23569 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35813 } }
I20260807 04:09:51.769860 23569 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:51.769980 23569 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e8f8c4be2d4c7ab767e9d0f724247a, State: Initialized, Role: FOLLOWER
I20260807 04:09:51.770794 23569 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35813 } }
I20260807 04:09:51.771117 23569 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:51.771286 23569 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:51.771489 23569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:51.773802 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:51.747077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.18.193.62:35813)
0807 04:09:51.747976 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:51.747988 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:51.748032 (+ 44us) server_negotiation.cc:407] Connection header received
0807 04:09:51.750175 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:51.750204 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:51.750741 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:51.751114 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:51.753226 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:51.755296 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:51.757250 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:51.758022 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:51.758178 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:51.763465 (+ 5287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:51.763481 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:51.763506 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:51.763577 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:51.768277 (+ 4700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:51.769207 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:51.769217 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:51.769226 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:51.769497 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:51.770212 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:51.770218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:51.770221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:51.771049 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:51.771405 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:51.772089 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 04:09:51.772678 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":434,"threads_started":1}
I20260807 04:09:51.774305 23569 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35813 } }
I20260807 04:09:51.774861 23569 leader_election.cc:302] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9e8f8c4be2d4c7ab767e9d0f724247a; no voters:
I20260807 04:09:51.775640 23569 leader_election.cc:288] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:51.775772 23574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:51.777115 23574 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [term 1 LEADER]: Becoming Leader. State: Replica: a9e8f8c4be2d4c7ab767e9d0f724247a, State: Running, Role: LEADER
I20260807 04:09:51.777706 23574 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35813 } }
I20260807 04:09:51.778280 23569 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:51.780848 23576 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9e8f8c4be2d4c7ab767e9d0f724247a. Latest consensus state: current_term: 1 leader_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35813 } } }
I20260807 04:09:51.781244 23576 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:51.780936 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e8f8c4be2d4c7ab767e9d0f724247a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35813 } } }
I20260807 04:09:51.781423 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:51.784538 23579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:51.790571 23579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:51.800252 23579 catalog_manager.cc:1347] Generated new cluster ID: a1fff8757ea1473996aec090549f4e1f
I20260807 04:09:51.800437 23579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:51.831189 23579 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:51.832887 23579 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:51.849308 23579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a: Generated new TSK 0
I20260807 04:09:51.850310 23579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:09:51.920701 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35813
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:52.289093 23595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:52.289451 23595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:52.289626 23595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:52.297629 23595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:52.297789 23595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:52.297961 23595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:52.308158 23595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:52.309666 23595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:52.311033 23595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:52.324649 23603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:52.324676 23601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:52.325083 23600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:52.326063 23595 server_base.cc:1034] running on GCE node
I20260807 04:09:52.326741 23595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:52.327903 23595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:52.329137 23595 hybrid_clock.cc:648] HybridClock initialized: now 1786075792329079 us; error 52 us; skew 500 ppm
I20260807 04:09:52.329473 23595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:52 dist-test-slave-8n49 krb5kdc[23486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075792, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:52.332888 23595 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:09:52.334113 23595 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:52.334249 23595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:52.334568 23595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:52.337906 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "1b9e972306b14672afcf3091cbe926c2"
format_stamp: "Formatted at 2026-08-07 04:09:52 on dist-test-slave-8n49"
server_key: "1c5f57ab797496e5e369fb9e3fb2b9e7"
server_key_iv: "c291c46a651e618284378efb73989cee"
server_key_version: "encryptionkey@0"
I20260807 04:09:52.338788 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "1b9e972306b14672afcf3091cbe926c2"
format_stamp: "Formatted at 2026-08-07 04:09:52 on dist-test-slave-8n49"
server_key: "1c5f57ab797496e5e369fb9e3fb2b9e7"
server_key_iv: "c291c46a651e618284378efb73989cee"
server_key_version: "encryptionkey@0"
I20260807 04:09:52.344290 23595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:09:52.348451 23610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:52.350139 23595 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:09:52.350349 23595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "1b9e972306b14672afcf3091cbe926c2"
format_stamp: "Formatted at 2026-08-07 04:09:52 on dist-test-slave-8n49"
server_key: "1c5f57ab797496e5e369fb9e3fb2b9e7"
server_key_iv: "c291c46a651e618284378efb73989cee"
server_key_version: "encryptionkey@0"
I20260807 04:09:52.350608 23595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:52.367247 23595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:52.372964 23595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:52.373286 23595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:52.374302 23595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:52.375731 23595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:52.375839 23595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:52.375954 23595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:52.376008 23595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:52.408048 23595 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:40049
I20260807 04:09:52.408125 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:40049 every 8 connection(s)
I20260807 04:09:52.409029 23595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:52.417315 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 23595
I20260807 04:09:52.417626 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:52.418329 19204 external_mini_cluster.cc:1428] Setting key 36757d81535ebccfc943d1b4159893cd
Aug 07 04:09:52 dist-test-slave-8n49 krb5kdc[23486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075792, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:52.432032 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:52.411713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:46687 (local address 127.18.193.62:35813)
0807 04:09:52.412193 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:52.412198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:52.413195 (+ 997us) server_negotiation.cc:407] Connection header received
0807 04:09:52.415236 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:52.415242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:52.415342 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:52.415481 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:52.417453 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.418306 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.420544 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.421006 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.421115 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:52.425717 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:52.425722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:52.425726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:52.425764 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:52.428390 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:52.429206 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:52.429217 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:52.429219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:52.429337 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:52.429778 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:52.429786 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:52.429792 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:52.430200 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:52.430366 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:52.431263 (+ 897us) server_negotiation.cc:299] Negotiation successful
0807 04:09:52.431531 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":268,"threads_started":1}
I20260807 04:09:52.432819 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:52.412119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35813 (local address 127.18.193.1:46687)
0807 04:09:52.412996 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:52.413037 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:52.414940 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:52.415724 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:52.415734 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:52.416312 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:52.417202 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:52.417234 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.418488 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.418496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:52.420389 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:52.420410 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.421280 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.421289 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:52.421651 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:52.422434 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:52.422480 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:52.425462 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:52.428595 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:52.428606 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:52.428624 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:52.429074 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:52.429446 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:52.429453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:52.429456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:52.429657 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:52.430368 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:52.430374 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:52.430824 (+ 450us) client_negotiation.cc:769] Sending connection context
0807 04:09:52.431113 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 04:09:52.431531 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":402,"threads_started":1}
I20260807 04:09:52.434481 23727 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35813
I20260807 04:09:52.434806 23727 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:52.435628 23727 heartbeater.cc:511] Master 127.18.193.62:35813 requested a full tablet report, sending...
I20260807 04:09:52.437948 23531 ts_manager.cc:194] Registered new tserver with Master: 1b9e972306b14672afcf3091cbe926c2 (127.18.193.1:40049)
I20260807 04:09:52.439500 23531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:46687
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:09:52.477833 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35813
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:52.836345 23733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:52.836699 23733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:52.836882 23733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:52.844527 23733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:52.844754 23733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:52.845010 23733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:52.855871 23733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:52.857362 23733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:52.858733 23733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:52.872601 23739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:52.872671 23741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:52.872644 23738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:52.873773 23733 server_base.cc:1034] running on GCE node
I20260807 04:09:52.874579 23733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:52.875581 23733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:52.876799 23733 hybrid_clock.cc:648] HybridClock initialized: now 1786075792876755 us; error 35 us; skew 500 ppm
I20260807 04:09:52.877147 23733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:52 dist-test-slave-8n49 krb5kdc[23486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075792, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:52.880532 23733 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:09:52.881814 23733 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:52.881906 23733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:52.882177 23733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:52.885351 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "ccba3afea0ad44ae9d3659270c11dd1d"
format_stamp: "Formatted at 2026-08-07 04:09:52 on dist-test-slave-8n49"
server_key: "b70e991f21c2be89b7d260eb31c72d32"
server_key_iv: "6f72827bcd07374851d6955a96b04fbe"
server_key_version: "encryptionkey@0"
I20260807 04:09:52.886323 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "ccba3afea0ad44ae9d3659270c11dd1d"
format_stamp: "Formatted at 2026-08-07 04:09:52 on dist-test-slave-8n49"
server_key: "b70e991f21c2be89b7d260eb31c72d32"
server_key_iv: "6f72827bcd07374851d6955a96b04fbe"
server_key_version: "encryptionkey@0"
I20260807 04:09:52.891603 23733 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 04:09:52.895358 23747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:52.897133 23733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:09:52.897301 23733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "ccba3afea0ad44ae9d3659270c11dd1d"
format_stamp: "Formatted at 2026-08-07 04:09:52 on dist-test-slave-8n49"
server_key: "b70e991f21c2be89b7d260eb31c72d32"
server_key_iv: "6f72827bcd07374851d6955a96b04fbe"
server_key_version: "encryptionkey@0"
I20260807 04:09:52.897576 23733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:52.913540 23733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:52.917837 23733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:52.918080 23733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:52.919000 23733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:52.920395 23733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:52.920490 23733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:52.920630 23733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:52.920709 23733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:52.958120 23733 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:44019
I20260807 04:09:52.958189 23865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:44019 every 8 connection(s)
I20260807 04:09:52.959034 23733 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:52.963668 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 23733
I20260807 04:09:52.963897 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:52.964542 19204 external_mini_cluster.cc:1428] Setting key 9d24b3350be894a39df84ac11bed0718
Aug 07 04:09:52 dist-test-slave-8n49 krb5kdc[23486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075792, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:52.984892 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:52.961685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:33391 (local address 127.18.193.62:35813)
0807 04:09:52.962191 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:52.962198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:52.963504 (+ 1306us) server_negotiation.cc:407] Connection header received
0807 04:09:52.966050 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:52.966055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:52.966127 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:52.966224 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:52.968383 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.969222 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.971817 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.972305 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.972465 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:52.977140 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:52.977149 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:52.977157 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:52.977207 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:52.981181 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:52.982007 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:52.982015 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:52.982021 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:52.982196 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:52.982681 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:52.982690 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:52.982696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:52.983121 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:52.983291 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:52.984059 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 04:09:52.984348 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":264,"threads_started":1}
I20260807 04:09:52.985895 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:52.962477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35813 (local address 127.18.193.2:33391)
0807 04:09:52.963282 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:52.963346 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:52.965664 (+ 2318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:52.966548 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:52.966568 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:52.967297 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:52.968157 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:52.968188 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.969390 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.969399 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:52.971625 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:52.971648 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:52.972618 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:52.972629 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:52.972960 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:52.973774 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:52.973815 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:52.976884 (+ 3069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:52.981352 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:52.981360 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:52.981375 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:52.981860 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:52.982346 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:52.982350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:52.982353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:52.982553 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:52.983280 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:52.983287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:52.983663 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 04:09:52.983918 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 04:09:52.984456 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":288,"threads_started":1}
I20260807 04:09:52.987535 23866 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35813
I20260807 04:09:52.987855 23866 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:52.988615 23866 heartbeater.cc:511] Master 127.18.193.62:35813 requested a full tablet report, sending...
I20260807 04:09:52.990190 23531 ts_manager.cc:194] Registered new tserver with Master: ccba3afea0ad44ae9d3659270c11dd1d (127.18.193.2:44019)
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
I20260807 04:09:52.991109 23531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:33391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:09:53.025198 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35813
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:09:53.363808 23874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:53.364120 23874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:53.364238 23874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:53.374239 23874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:53.374372 23874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:53.374581 23874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:53.386126 23874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:53.387552 23874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:53.389039 23874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:53.402647 23882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:53.402594 23879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:53.402695 23880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:53.406499 23874 server_base.cc:1034] running on GCE node
I20260807 04:09:53.407298 23874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:53.408700 23874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:53.409962 23874 hybrid_clock.cc:648] HybridClock initialized: now 1786075793409890 us; error 62 us; skew 500 ppm
I20260807 04:09:53.410341 23874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:09:53 dist-test-slave-8n49 krb5kdc[23486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075793, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:09:53.414500 23874 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:09:53.415802 23874 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:53.415940 23874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:53.416265 23874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:53.419375 23874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "a24cc6845c4249deb75c29c2a4e774b0"
format_stamp: "Formatted at 2026-08-07 04:09:53 on dist-test-slave-8n49"
server_key: "a9729bf0f9098ea04e9055742efee2a9"
server_key_iv: "586c93b06c52fc4cb8fbbf7baf863ef0"
server_key_version: "encryptionkey@0"
I20260807 04:09:53.420233 23874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "a24cc6845c4249deb75c29c2a4e774b0"
format_stamp: "Formatted at 2026-08-07 04:09:53 on dist-test-slave-8n49"
server_key: "a9729bf0f9098ea04e9055742efee2a9"
server_key_iv: "586c93b06c52fc4cb8fbbf7baf863ef0"
server_key_version: "encryptionkey@0"
I20260807 04:09:53.425580 23874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:09:53.429361 23889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:53.430711 23874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:09:53.430898 23874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "a24cc6845c4249deb75c29c2a4e774b0"
format_stamp: "Formatted at 2026-08-07 04:09:53 on dist-test-slave-8n49"
server_key: "a9729bf0f9098ea04e9055742efee2a9"
server_key_iv: "586c93b06c52fc4cb8fbbf7baf863ef0"
server_key_version: "encryptionkey@0"
I20260807 04:09:53.431154 23874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:53.443562 23727 heartbeater.cc:503] Master 127.18.193.62:35813 was elected leader, sending a full tablet report...
I20260807 04:09:53.444412 23874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:53.449023 23874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:53.449298 23874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:53.450256 23874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:53.451671 23874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:53.451799 23874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:53.451920 23874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:53.452010 23874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:53.487218 23874 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:38981
I20260807 04:09:53.487270 24005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:38981 every 8 connection(s)
I20260807 04:09:53.488487 23874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:53.489241 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 23874
I20260807 04:09:53.489464 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:53.490101 19204 external_mini_cluster.cc:1428] Setting key 8358b1dad323a48a64ba7f5e04d4c883
Aug 07 04:09:53 dist-test-slave-8n49 krb5kdc[23486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075793, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:09:53.509582 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.491236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:45287 (local address 127.18.193.62:35813)
0807 04:09:53.491746 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.491752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.493032 (+ 1280us) server_negotiation.cc:407] Connection header received
0807 04:09:53.495089 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.495093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.495162 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.495252 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:53.496739 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.497499 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.499051 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.499463 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.499561 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.503652 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:53.503657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:53.503661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:53.503696 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:53.506764 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:53.507489 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:53.507493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:53.507495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:53.507610 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:53.507997 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:53.508000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:53.508002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:53.508263 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:53.508379 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.509148 (+ 769us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.509307 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":220,"threads_started":1}
I20260807 04:09:53.510531 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.491882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35813 (local address 127.18.193.3:45287)
0807 04:09:53.492829 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:53.492891 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:53.494838 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:53.495451 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:53.495459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:53.495970 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:53.496534 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.496600 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.497649 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.497652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.498906 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.498919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.499664 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.499668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.499848 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.500894 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:09:53.500923 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:53.503426 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:53.506921 (+ 3495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:53.506928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:53.506942 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:53.507377 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:53.507702 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:09:53.507706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:09:53.507708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:09:53.507892 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:09:53.508368 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:53.508374 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:53.508788 (+ 414us) client_negotiation.cc:769] Sending connection context
0807 04:09:53.509030 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 04:09:53.509415 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":397,"threads_started":1}
I20260807 04:09:53.511828 24006 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35813
I20260807 04:09:53.512151 24006 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:53.512885 24006 heartbeater.cc:511] Master 127.18.193.62:35813 requested a full tablet report, sending...
I20260807 04:09:53.514366 23531 ts_manager.cc:194] Registered new tserver with Master: a24cc6845c4249deb75c29c2a4e774b0 (127.18.193.3:38981)
I20260807 04:09:53.515291 23531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:45287
I20260807 04:09:53.518867 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:09:53.536834 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.523792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56166 (local address 127.18.193.62:35813)
0807 04:09:53.523979 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.523983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.524534 (+ 551us) server_negotiation.cc:407] Connection header received
0807 04:09:53.524756 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.524760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.524830 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.524961 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:53.526313 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.527080 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.528373 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.528832 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.528956 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.531274 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:53.531279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:53.531282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:53.531310 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:53.534115 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:53.534737 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:53.534740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:53.534743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:53.534857 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:53.535225 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:53.535228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:53.535230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:53.535597 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:53.535835 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.536265 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.536442 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:09:53.541680 23531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:53.548851 23531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:53.586143 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.564860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54958 (local address 127.18.193.2:44019)
0807 04:09:53.577062 (+ 12202us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.577071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.578420 (+ 1349us) server_negotiation.cc:407] Connection header received
0807 04:09:53.578795 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.578800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.579049 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.579252 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:53.579913 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.581376 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.583459 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.584904 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.585058 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.585261 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.585595 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.585776 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12055,"thread_start_us":11916,"threads_started":1}
I20260807 04:09:53.586943 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.562618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:38981 (local address 127.0.0.1:45714)
0807 04:09:53.576280 (+ 13662us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:53.576338 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:53.576713 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:53.578008 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:53.578015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:53.578062 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:53.578597 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.578613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.580749 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.580754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.583687 (+ 2933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.583703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.585988 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.585995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.586171 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.586211 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 04:09:53.586317 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 04:09:53.586454 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13298,"mutex_wait_us":53,"thread_start_us":13127,"threads_started":1}
I20260807 04:09:53.587579 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.563467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44019 (local address 127.0.0.1:54958)
0807 04:09:53.578301 (+ 14834us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:53.578319 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:53.578607 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:53.579260 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:53.579267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:53.579296 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:53.579689 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.579722 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.581526 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.581534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.583272 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.583286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.585117 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.585124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.585330 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.585383 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 04:09:53.585546 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 04:09:53.587120 (+ 1574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14680,"mutex_wait_us":64,"thread_start_us":13734,"threads_started":1}
I20260807 04:09:53.587677 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.562961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:40049 (local address 127.0.0.1:57342)
0807 04:09:53.577503 (+ 14542us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:53.577524 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:53.577683 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:53.578332 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:53.578336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:53.578379 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:53.578941 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.578951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.581055 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.581061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.583687 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.583703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.586827 (+ 3124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.586835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.587025 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.587058 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:09:53.587146 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 04:09:53.587278 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14421,"mutex_wait_us":64,"thread_start_us":15024,"threads_started":1}
I20260807 04:09:53.587880 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.563101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.18.193.3:38981)
0807 04:09:53.575348 (+ 12247us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.575358 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.576474 (+ 1116us) server_negotiation.cc:407] Connection header received
0807 04:09:53.577034 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.577044 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.577324 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.577547 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:53.578911 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.580577 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.583916 (+ 3339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.585490 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.585632 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.585837 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.587404 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.587528 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12015,"thread_start_us":12259,"threads_started":1}
I20260807 04:09:53.588316 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.564244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57342 (local address 127.18.193.1:40049)
0807 04:09:53.577277 (+ 13033us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.577285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.577637 (+ 352us) server_negotiation.cc:407] Connection header received
0807 04:09:53.577857 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.577867 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.578116 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.578311 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:53.579107 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.580823 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.584715 (+ 3892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.586291 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.586444 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.586671 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.587880 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.588059 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12851,"thread_start_us":12740,"threads_started":1}
I20260807 04:09:53.591369 23798 tablet_service.cc:1467] Processing CreateTablet for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 (DEFAULT_TABLE table=test-table [id=e662c038bf9c47ddbc767e894e427729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:53.591809 23660 tablet_service.cc:1467] Processing CreateTablet for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 (DEFAULT_TABLE table=test-table [id=e662c038bf9c47ddbc767e894e427729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:53.592736 23798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:53.593200 23660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:53.595340 23938 tablet_service.cc:1467] Processing CreateTablet for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 (DEFAULT_TABLE table=test-table [id=e662c038bf9c47ddbc767e894e427729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:53.596889 23938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:53.605450 24025 tablet_bootstrap.cc:492] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Bootstrap starting.
I20260807 04:09:53.608825 24025 tablet_bootstrap.cc:654] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:53.609342 24026 tablet_bootstrap.cc:492] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Bootstrap starting.
I20260807 04:09:53.610316 24025 log.cc:824] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:53.612968 24028 tablet_bootstrap.cc:492] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Bootstrap starting.
I20260807 04:09:53.613080 24026 tablet_bootstrap.cc:654] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:53.613809 24025 tablet_bootstrap.cc:492] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: No bootstrap required, opened a new log
I20260807 04:09:53.614110 24025 ts_tablet_manager.cc:1397] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260807 04:09:53.614374 24026 log.cc:824] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:53.616171 24028 tablet_bootstrap.cc:654] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:53.617424 24028 log.cc:824] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:53.618266 24026 tablet_bootstrap.cc:492] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: No bootstrap required, opened a new log
I20260807 04:09:53.618575 24026 ts_tablet_manager.cc:1397] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 04:09:53.620213 24028 tablet_bootstrap.cc:492] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: No bootstrap required, opened a new log
I20260807 04:09:53.620570 24028 ts_tablet_manager.cc:1397] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260807 04:09:53.625262 24025 raft_consensus.cc:348] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.625765 24025 raft_consensus.cc:374] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:53.625931 24025 raft_consensus.cc:729] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccba3afea0ad44ae9d3659270c11dd1d, State: Initialized, Role: FOLLOWER
I20260807 04:09:53.626617 24025 consensus_queue.cc:260] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.627852 23866 heartbeater.cc:503] Master 127.18.193.62:35813 was elected leader, sending a full tablet report...
I20260807 04:09:53.628607 24028 raft_consensus.cc:348] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.629135 24028 raft_consensus.cc:374] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:53.629290 24028 raft_consensus.cc:729] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24cc6845c4249deb75c29c2a4e774b0, State: Initialized, Role: FOLLOWER
I20260807 04:09:53.629242 24026 raft_consensus.cc:348] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.629726 24026 raft_consensus.cc:374] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:53.629868 24026 raft_consensus.cc:729] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9e972306b14672afcf3091cbe926c2, State: Initialized, Role: FOLLOWER
I20260807 04:09:53.630153 24028 consensus_queue.cc:260] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.630548 24026 consensus_queue.cc:260] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.631309 24006 heartbeater.cc:503] Master 127.18.193.62:35813 was elected leader, sending a full tablet report...
I20260807 04:09:53.632061 24028 ts_tablet_manager.cc:1428] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260807 04:09:53.633738 24026 ts_tablet_manager.cc:1428] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260807 04:09:53.634716 24025 ts_tablet_manager.cc:1428] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Time spent starting tablet: real 0.020s user 0.011s sys 0.003s
W20260807 04:09:53.661547 23728 tablet.cc:2369] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:09:53.711266 23867 tablet.cc:2369] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:09:53.740168 24007 tablet.cc:2369] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:09:53.788650 24032 raft_consensus.cc:482] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:53.788988 24032 raft_consensus.cc:504] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.790792 24032 leader_election.cc:288] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b9e972306b14672afcf3091cbe926c2 (127.18.193.1:40049), ccba3afea0ad44ae9d3659270c11dd1d (127.18.193.2:44019)
I20260807 04:09:53.797706 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.790983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:40049 (local address 127.18.193.3:34489)
0807 04:09:53.791232 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:53.791255 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:53.791396 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:53.791896 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:53.791911 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:53.791943 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:53.792180 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.792194 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.793580 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.793585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.795567 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.795584 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.797098 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.797103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.797235 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.797260 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:09:53.797339 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 04:09:53.797439 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 04:09:53.797820 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.791177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:34489 (local address 127.18.193.1:40049)
0807 04:09:53.791387 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.791394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.791420 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:09:53.791574 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.791582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.791685 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.791806 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:53.792340 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.793419 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.795765 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.796902 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.797074 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.797158 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.797398 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.797529 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:09:53.798549 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.791570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:52369 (local address 127.18.193.2:44019)
0807 04:09:53.791787 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.791794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.792305 (+ 511us) server_negotiation.cc:407] Connection header received
0807 04:09:53.792452 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.792458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.792546 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.792668 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:09:53.793107 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.794024 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.796231 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.797505 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.797663 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.797731 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.798146 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.798291 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:09:53.798640 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.791336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44019 (local address 127.18.193.3:52369)
0807 04:09:53.792163 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:53.792187 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:53.792311 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:53.792679 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:53.792682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:53.792701 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:09:53.792953 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.792960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.794188 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.794194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.796060 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:53.796070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.797723 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.797732 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:53.797920 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.797957 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 04:09:53.798114 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 04:09:53.798245 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":4,"thread_start_us":461,"threads_started":1}
I20260807 04:09:53.798897 23680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b6e6c25fc94561b4a3f7c2c31bbac7" candidate_uuid: "a24cc6845c4249deb75c29c2a4e774b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9e972306b14672afcf3091cbe926c2" is_pre_election: true
I20260807 04:09:53.799546 23680 raft_consensus.cc:2454] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24cc6845c4249deb75c29c2a4e774b0 in term 0.
I20260807 04:09:53.799772 23818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b6e6c25fc94561b4a3f7c2c31bbac7" candidate_uuid: "a24cc6845c4249deb75c29c2a4e774b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" is_pre_election: true
I20260807 04:09:53.800316 23893 leader_election.cc:302] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b9e972306b14672afcf3091cbe926c2, a24cc6845c4249deb75c29c2a4e774b0; no voters:
I20260807 04:09:53.800321 23818 raft_consensus.cc:2454] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24cc6845c4249deb75c29c2a4e774b0 in term 0.
I20260807 04:09:53.800768 24032 raft_consensus.cc:2781] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:53.800949 24032 raft_consensus.cc:482] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:53.801090 24032 raft_consensus.cc:3037] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:53.803700 24032 raft_consensus.cc:504] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.804502 24032 leader_election.cc:288] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [CANDIDATE]: Term 1 election: Requested vote from peers 1b9e972306b14672afcf3091cbe926c2 (127.18.193.1:40049), ccba3afea0ad44ae9d3659270c11dd1d (127.18.193.2:44019)
I20260807 04:09:53.805068 23680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b6e6c25fc94561b4a3f7c2c31bbac7" candidate_uuid: "a24cc6845c4249deb75c29c2a4e774b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9e972306b14672afcf3091cbe926c2"
I20260807 04:09:53.805102 23818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b6e6c25fc94561b4a3f7c2c31bbac7" candidate_uuid: "a24cc6845c4249deb75c29c2a4e774b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccba3afea0ad44ae9d3659270c11dd1d"
I20260807 04:09:53.805312 23680 raft_consensus.cc:3037] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:53.805312 23818 raft_consensus.cc:3037] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:53.808710 23818 raft_consensus.cc:2454] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24cc6845c4249deb75c29c2a4e774b0 in term 1.
I20260807 04:09:53.808710 23680 raft_consensus.cc:2454] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24cc6845c4249deb75c29c2a4e774b0 in term 1.
I20260807 04:09:53.809181 23890 leader_election.cc:302] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24cc6845c4249deb75c29c2a4e774b0, ccba3afea0ad44ae9d3659270c11dd1d; no voters:
I20260807 04:09:53.809494 24032 raft_consensus.cc:2781] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:53.810001 24032 raft_consensus.cc:686] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 1 LEADER]: Becoming Leader. State: Replica: a24cc6845c4249deb75c29c2a4e774b0, State: Running, Role: LEADER
I20260807 04:09:53.810652 24032 consensus_queue.cc:237] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } }
I20260807 04:09:53.815372 23530 catalog_manager.cc:5495] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a24cc6845c4249deb75c29c2a4e774b0 (127.18.193.3). New cstate: current_term: 1 leader_uuid: "a24cc6845c4249deb75c29c2a4e774b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24cc6845c4249deb75c29c2a4e774b0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 38981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 } health_report { overall_health: UNKNOWN } } }
I20260807 04:09:53.875710 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.869527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.18.193.3:38981)
0807 04:09:53.869744 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.869749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.869766 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:09:53.869990 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.869995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.870079 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.870212 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:09:53.870697 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.871490 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.873243 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.873649 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.873753 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.874290 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:09:53.874950 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:09:53.875081 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.875279 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.875355 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 04:09:53.884654 23680 raft_consensus.cc:1261] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 1 FOLLOWER]: Refusing update from remote peer a24cc6845c4249deb75c29c2a4e774b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:53.884915 23818 raft_consensus.cc:1261] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 1 FOLLOWER]: Refusing update from remote peer a24cc6845c4249deb75c29c2a4e774b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:53.885898 24035 consensus_queue.cc:1035] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b9e972306b14672afcf3091cbe926c2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 40049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:53.886291 24032 consensus_queue.cc:1035] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccba3afea0ad44ae9d3659270c11dd1d" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:53.896817 24039 mvcc.cc:205] Tried to move back new op lower bound from 7315766451740151808 to 7315766451452428288. Current Snapshot: MvccSnapshot[applied={T|T < 7315766451740151808}]
I20260807 04:09:53.918527 23530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:53.918915 23530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:53.923045 23530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9e8f8c4be2d4c7ab767e9d0f724247a: Sending DeleteTablet for 3 replicas of tablet b5b6e6c25fc94561b4a3f7c2c31bbac7
I20260807 04:09:53.924279 23938 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:53 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:45714
I20260807 04:09:53.924419 23660 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:53 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:57342
I20260807 04:09:53.924419 23798 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:53 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:54958
I20260807 04:09:53.925283 24050 tablet_replica.cc:330] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: stopping tablet replica
I20260807 04:09:53.925810 24050 raft_consensus.cc:2229] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:09:53.926661 24050 raft_consensus.cc:2258] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:53.928097 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 23595
I20260807 04:09:53.929418 24050 ts_tablet_manager.cc:1905] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:53.937248 24052 tablet_replica.cc:330] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: stopping tablet replica
I20260807 04:09:53.937587 24050 ts_tablet_manager.cc:1918] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:53.937821 24050 log.cc:1196] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/b5b6e6c25fc94561b4a3f7c2c31bbac7
I20260807 04:09:53.938117 24052 raft_consensus.cc:2229] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:53.938469 24050 ts_tablet_manager.cc:1939] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P a24cc6845c4249deb75c29c2a4e774b0: Deleting consensus metadata
I20260807 04:09:53.938807 24052 raft_consensus.cc:2258] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:53.940505 23517 catalog_manager.cc:4841] TS a24cc6845c4249deb75c29c2a4e774b0 (127.18.193.3:38981): tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 (table test-table [id=e662c038bf9c47ddbc767e894e427729]) successfully deleted
I20260807 04:09:53.941308 24052 ts_tablet_manager.cc:1905] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:53.941840 24051 tablet_replica.cc:330] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: stopping tablet replica
I20260807 04:09:53.942376 24051 raft_consensus.cc:2229] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:53.942941 24051 raft_consensus.cc:2258] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:53.945541 24051 ts_tablet_manager.cc:1905] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:53.948891 24052 ts_tablet_manager.cc:1918] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:53.949561 24052 log.cc:1196] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/b5b6e6c25fc94561b4a3f7c2c31bbac7
I20260807 04:09:53.950233 24052 ts_tablet_manager.cc:1939] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P ccba3afea0ad44ae9d3659270c11dd1d: Deleting consensus metadata
I20260807 04:09:53.952493 23515 catalog_manager.cc:4841] TS ccba3afea0ad44ae9d3659270c11dd1d (127.18.193.2:44019): tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 (table test-table [id=e662c038bf9c47ddbc767e894e427729]) successfully deleted
Aug 07 04:09:53 dist-test-slave-8n49 krb5kdc[23486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:09:53.953361 24051 ts_tablet_manager.cc:1918] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:53.953549 24051 log.cc:1196] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/b5b6e6c25fc94561b4a3f7c2c31bbac7
I20260807 04:09:53.954174 24051 ts_tablet_manager.cc:1939] T b5b6e6c25fc94561b4a3f7c2c31bbac7 P 1b9e972306b14672afcf3091cbe926c2: Deleting consensus metadata
I20260807 04:09:53.956207 23518 catalog_manager.cc:4841] TS 1b9e972306b14672afcf3091cbe926c2 (127.18.193.1:40049): tablet b5b6e6c25fc94561b4a3f7c2c31bbac7 (table test-table [id=e662c038bf9c47ddbc767e894e427729]) successfully deleted
I20260807 04:09:53.960729 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:53.929931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.18.193.1:40049)
0807 04:09:53.930185 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:53.930194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:53.941144 (+ 10950us) server_negotiation.cc:407] Connection header received
0807 04:09:53.941354 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:53.941360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:53.941465 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:53.941580 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:53.943446 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.944716 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.946614 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:53.948060 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:53.948175 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:53.953838 (+ 5663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:53.953856 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:53.953873 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:53.953913 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:53.957527 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:53.958330 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:53.958339 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:53.958343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:53.958505 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:53.958914 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:53.958920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:53.958923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:53.959309 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:53.959649 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:53.959943 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 04:09:53.960169 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:09:53.961475 23722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:54.056326 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 23595
I20260807 04:09:54.077845 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 23733
Aug 07 04:09:54 dist-test-slave-8n49 krb5kdc[23486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:09:54.095500 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:54.078890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.18.193.2:44019)
0807 04:09:54.079158 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:54.079164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:54.079185 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:09:54.079267 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:54.079273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:54.079353 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:54.079470 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:54.080832 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:54.081632 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:54.082931 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:54.083365 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:54.083510 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:54.088308 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:54.088339 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:54.088356 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:54.088411 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:54.091618 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:54.092952 (+ 1334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:54.092961 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:54.092963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:54.093120 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:54.093650 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:54.093654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:54.093656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:54.094147 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:54.094452 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:54.094866 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 04:09:54.095125 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 04:09:54.096315 23861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:54.189059 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 23733
I20260807 04:09:54.208943 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 23874
Aug 07 04:09:54 dist-test-slave-8n49 krb5kdc[23486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:09:54.227059 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:54.209777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.18.193.3:38981)
0807 04:09:54.209988 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:54.209994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:54.210018 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:09:54.210147 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:54.210154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:54.210243 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:54.210374 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:54.211640 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:54.212950 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:54.214391 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:54.215114 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:54.215260 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:54.220244 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:54.220257 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:54.220275 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:09:54.220320 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:54.223735 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:54.224677 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:54.224685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:54.224687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:54.224853 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:09:54.225318 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:09:54.225321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:09:54.225323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:09:54.225686 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:54.225922 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:54.226445 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 04:09:54.226616 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:09:54.227823 24001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:54.352169 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 23874
I20260807 04:09:54.373589 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 23500
I20260807 04:09:54.374835 23564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:54.488130 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 23500
2026-08-07T04:09:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4649 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T04:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:54Z Disabled control of system clock
I20260807 04:09:54.565666 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:42697
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:42697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 04:09:54.916297 24061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:54.916702 24061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:54.916848 24061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:54.924969 24061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:54.925089 24061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:54.925134 24061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:54.925197 24061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:54.925238 24061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:09:54.936251 24061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:42697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:42697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:54.937819 24061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:54.939764 24061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:54.952764 24066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:54.952729 24067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:54.952684 24070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:54.953859 24061 server_base.cc:1034] running on GCE node
I20260807 04:09:54.954680 24061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:54.956101 24061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:54.957408 24061 hybrid_clock.cc:648] HybridClock initialized: now 1786075794957301 us; error 59 us; skew 500 ppm
I20260807 04:09:54.957844 24061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:09:54.960196 24061 webserver.cc:460] Webserver started at http://127.18.193.62:33887/ using document root <none> and password file <none>
I20260807 04:09:54.961054 24061 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:54.961175 24061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:54.961520 24061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:54.964495 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "54d4b18e1ec3415f9beb857325ace499"
format_stamp: "Formatted at 2026-08-07 04:09:54 on dist-test-slave-8n49"
server_key: "ddcb7070ed0c0a78a31aa9508d81e684"
server_key_iv: "c17f82e458774c816cad8734813476af"
server_key_version: "encryptionkey@0"
I20260807 04:09:54.965456 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "54d4b18e1ec3415f9beb857325ace499"
format_stamp: "Formatted at 2026-08-07 04:09:54 on dist-test-slave-8n49"
server_key: "ddcb7070ed0c0a78a31aa9508d81e684"
server_key_iv: "c17f82e458774c816cad8734813476af"
server_key_version: "encryptionkey@0"
I20260807 04:09:54.970880 24061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:09:54.974471 24076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:54.975814 24061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:09:54.975993 24061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "54d4b18e1ec3415f9beb857325ace499"
format_stamp: "Formatted at 2026-08-07 04:09:54 on dist-test-slave-8n49"
server_key: "ddcb7070ed0c0a78a31aa9508d81e684"
server_key_iv: "c17f82e458774c816cad8734813476af"
server_key_version: "encryptionkey@0"
I20260807 04:09:54.976236 24061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:54.982131 24061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:54.982928 24061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:54.983230 24061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:55.001950 24061 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:42697
I20260807 04:09:55.001991 24128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:42697 every 8 connection(s)
I20260807 04:09:55.003391 24061 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:55.007382 24131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:55.008721 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24061
I20260807 04:09:55.008914 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:55.009632 19204 external_mini_cluster.cc:1428] Setting key f7e15a5ac72620528930837aa7abccae
I20260807 04:09:55.019057 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:55.011822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.18.193.62:42697)
0807 04:09:55.013048 (+ 1226us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:55.013063 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:55.013118 (+ 55us) server_negotiation.cc:407] Connection header received
0807 04:09:55.014866 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:55.014895 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:55.015487 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:55.015942 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:55.016988 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:55.016997 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:55.017013 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:55.017031 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:55.017113 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:55.017138 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:55.017372 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:55.017514 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 04:09:55.017790 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":377,"threads_started":1}
I20260807 04:09:55.022554 24131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499: Bootstrap starting.
I20260807 04:09:55.025629 24131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:55.026607 24131 log.cc:824] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:55.029611 24131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499: No bootstrap required, opened a new log
I20260807 04:09:55.038563 24131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d4b18e1ec3415f9beb857325ace499" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42697 } }
I20260807 04:09:55.038980 24131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:55.039090 24131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d4b18e1ec3415f9beb857325ace499, State: Initialized, Role: FOLLOWER
I20260807 04:09:55.039717 24131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d4b18e1ec3415f9beb857325ace499" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42697 } }
I20260807 04:09:55.039988 24131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:55.040153 24131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:55.040292 24131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:55.042532 24131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d4b18e1ec3415f9beb857325ace499" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42697 } }
I20260807 04:09:55.043012 24131 leader_election.cc:302] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d4b18e1ec3415f9beb857325ace499; no voters:
I20260807 04:09:55.043634 24131 leader_election.cc:288] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:55.043840 24136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:55.044647 24136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [term 1 LEADER]: Becoming Leader. State: Replica: 54d4b18e1ec3415f9beb857325ace499, State: Running, Role: LEADER
I20260807 04:09:55.045365 24136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d4b18e1ec3415f9beb857325ace499" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42697 } }
I20260807 04:09:55.046028 24131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:55.047988 24137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54d4b18e1ec3415f9beb857325ace499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d4b18e1ec3415f9beb857325ace499" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42697 } } }
I20260807 04:09:55.048360 24137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:55.049037 24138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d4b18e1ec3415f9beb857325ace499. Latest consensus state: current_term: 1 leader_uuid: "54d4b18e1ec3415f9beb857325ace499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d4b18e1ec3415f9beb857325ace499" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 42697 } } }
I20260807 04:09:55.049721 24138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:55.057787 24141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:55.062150 24141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:55.070791 24141 catalog_manager.cc:1347] Generated new cluster ID: ab01674201714faead97198db6144716
I20260807 04:09:55.070915 24141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:55.084347 24141 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:55.084872 24141 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:55.093204 24141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499: Generated new TSK 0
I20260807 04:09:55.094059 24141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:09:55.107416 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42697
--builtin_ntp_servers=127.18.193.20:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 04:09:55.487509 24156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:55.487807 24156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:55.487898 24156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:55.495409 24156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:55.495509 24156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:55.495625 24156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:09:55.504906 24156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:55.506295 24156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:55.508041 24156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:55.521533 24162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:55.521580 24161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:55.521814 24164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:55.522379 24156 server_base.cc:1034] running on GCE node
I20260807 04:09:55.523075 24156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:55.523989 24156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:55.525228 24156 hybrid_clock.cc:648] HybridClock initialized: now 1786075795525183 us; error 58 us; skew 500 ppm
I20260807 04:09:55.525565 24156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:09:55.527803 24156 webserver.cc:460] Webserver started at http://127.18.193.1:40649/ using document root <none> and password file <none>
I20260807 04:09:55.528630 24156 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:55.528725 24156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:55.529008 24156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:55.531834 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "017e9b775681494ba7d73b5875d0bc18"
format_stamp: "Formatted at 2026-08-07 04:09:55 on dist-test-slave-8n49"
server_key: "93e6740a95bfc3784acfe2a165654f52"
server_key_iv: "8ce6728e4fbacc376a22f329a9e703a0"
server_key_version: "encryptionkey@0"
I20260807 04:09:55.532682 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "017e9b775681494ba7d73b5875d0bc18"
format_stamp: "Formatted at 2026-08-07 04:09:55 on dist-test-slave-8n49"
server_key: "93e6740a95bfc3784acfe2a165654f52"
server_key_iv: "8ce6728e4fbacc376a22f329a9e703a0"
server_key_version: "encryptionkey@0"
I20260807 04:09:55.537847 24156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:09:55.541471 24170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:55.542858 24156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:09:55.543047 24156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "017e9b775681494ba7d73b5875d0bc18"
format_stamp: "Formatted at 2026-08-07 04:09:55 on dist-test-slave-8n49"
server_key: "93e6740a95bfc3784acfe2a165654f52"
server_key_iv: "8ce6728e4fbacc376a22f329a9e703a0"
server_key_version: "encryptionkey@0"
I20260807 04:09:55.543288 24156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:55.549371 24156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:55.550309 24156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:55.550719 24156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:55.552129 24156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:55.553869 24156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:55.553989 24156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:55.554136 24156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:55.554224 24156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:55.587215 24156 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:39807
I20260807 04:09:55.587271 24285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:39807 every 8 connection(s)
I20260807 04:09:55.588572 24156 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:09:55.596925 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:55.591174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:54249 (local address 127.18.193.62:42697)
0807 04:09:55.591685 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:55.591689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:55.592947 (+ 1258us) server_negotiation.cc:407] Connection header received
0807 04:09:55.594646 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:55.594650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:55.594705 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:55.594810 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:55.596121 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:55.596126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:55.596129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:55.596146 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:55.596189 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:55.596202 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:55.596307 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:55.596461 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 04:09:55.596512 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":267,"threads_started":1}
I20260807 04:09:55.597121 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24156
I20260807 04:09:55.597316 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:09:55.597796 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:55.591651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:42697 (local address 127.18.193.1:54249)
0807 04:09:55.592768 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:55.592818 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:55.594389 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:55.595015 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:55.595034 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:55.595121 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:55.595829 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:55.595832 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:55.595869 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:55.595952 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:55.596290 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:55.596297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:55.596303 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:09:55.596503 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 04:09:55.596757 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":372,"threads_started":1}
I20260807 04:09:55.598043 19204 external_mini_cluster.cc:1428] Setting key b9cc5e20bf95e95260e5c88b4f4f6578
I20260807 04:09:55.599361 24286 heartbeater.cc:347] Connected to a master server at 127.18.193.62:42697
I20260807 04:09:55.599684 24286 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:55.600371 24286 heartbeater.cc:511] Master 127.18.193.62:42697 requested a full tablet report, sending...
I20260807 04:09:55.600940 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42697
--builtin_ntp_servers=127.18.193.20:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 04:09:55.602562 24093 ts_manager.cc:194] Registered new tserver with Master: 017e9b775681494ba7d73b5875d0bc18 (127.18.193.1:39807)
W20260807 04:09:55.937515 24290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:55.937829 24290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:55.937975 24290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:55.945668 24290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:55.945792 24290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:55.945964 24290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:09:55.955982 24290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:55.957432 24290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:55.959277 24290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:55.973577 24296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:55.973637 24299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:55.973614 24297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:55.974298 24290 server_base.cc:1034] running on GCE node
I20260807 04:09:55.974949 24290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:55.976013 24290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:55.977245 24290 hybrid_clock.cc:648] HybridClock initialized: now 1786075795977198 us; error 43 us; skew 500 ppm
I20260807 04:09:55.977617 24290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:09:55.980130 24290 webserver.cc:460] Webserver started at http://127.18.193.2:43995/ using document root <none> and password file <none>
I20260807 04:09:55.981069 24290 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:55.981204 24290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:55.981575 24290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:55.984637 24290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8"
format_stamp: "Formatted at 2026-08-07 04:09:55 on dist-test-slave-8n49"
server_key: "481849956bc3be4e8abea836ed175693"
server_key_iv: "2a48f5be6c255931d17da45008ca219f"
server_key_version: "encryptionkey@0"
I20260807 04:09:55.985566 24290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8"
format_stamp: "Formatted at 2026-08-07 04:09:55 on dist-test-slave-8n49"
server_key: "481849956bc3be4e8abea836ed175693"
server_key_iv: "2a48f5be6c255931d17da45008ca219f"
server_key_version: "encryptionkey@0"
I20260807 04:09:55.991192 24290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:09:55.994833 24306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:55.996337 24290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:09:55.996528 24290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8"
format_stamp: "Formatted at 2026-08-07 04:09:55 on dist-test-slave-8n49"
server_key: "481849956bc3be4e8abea836ed175693"
server_key_iv: "2a48f5be6c255931d17da45008ca219f"
server_key_version: "encryptionkey@0"
I20260807 04:09:55.996845 24290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:56.003022 24290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:56.003856 24290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:56.004149 24290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:56.005291 24290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:56.006770 24290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:56.006904 24290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:56.007045 24290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:56.007135 24290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:56.045686 24290 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:41377
I20260807 04:09:56.045739 24421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:41377 every 8 connection(s)
I20260807 04:09:56.047243 24290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:09:56.054159 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24290
I20260807 04:09:56.054343 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:09:56.054994 19204 external_mini_cluster.cc:1428] Setting key 623263bf41e99464a094821cc73d7cb9
I20260807 04:09:56.056397 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.049985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:37663 (local address 127.18.193.62:42697)
0807 04:09:56.050178 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.050183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.051523 (+ 1340us) server_negotiation.cc:407] Connection header received
0807 04:09:56.053369 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.053373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.053444 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.053539 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.055040 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.055044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.055065 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.055109 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.055156 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.055177 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.055332 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.055968 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.056054 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:09:56.057207 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.050375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:42697 (local address 127.18.193.2:37663)
0807 04:09:56.051352 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.051395 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:56.053116 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:56.053772 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:56.053781 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:56.053869 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:56.054666 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:56.054671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:56.054720 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:56.054841 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:56.055310 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:56.055321 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:56.055329 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 04:09:56.055730 (+ 401us) client_negotiation.cc:240] Negotiation successful
0807 04:09:56.056025 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":269,"threads_started":1}
I20260807 04:09:56.058439 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42697
--builtin_ntp_servers=127.18.193.20:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 04:09:56.058784 24422 heartbeater.cc:347] Connected to a master server at 127.18.193.62:42697
I20260807 04:09:56.059104 24422 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:56.059693 24422 heartbeater.cc:511] Master 127.18.193.62:42697 requested a full tablet report, sending...
I20260807 04:09:56.061136 24093 ts_manager.cc:194] Registered new tserver with Master: 0fd7e2b66b9b4d18bceb19b4addd45d8 (127.18.193.2:41377)
W20260807 04:09:56.401055 24425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:56.401417 24425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:56.401516 24425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:56.408913 24425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:56.409034 24425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:56.409148 24425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:09:56.418637 24425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:42697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:56.419983 24425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:56.421831 24425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:56.435642 24434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:56.435711 24432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:56.435667 24431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:56.436674 24425 server_base.cc:1034] running on GCE node
I20260807 04:09:56.437353 24425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:56.438411 24425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:56.439685 24425 hybrid_clock.cc:648] HybridClock initialized: now 1786075796439594 us; error 91 us; skew 500 ppm
I20260807 04:09:56.440088 24425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:09:56.442885 24425 webserver.cc:460] Webserver started at http://127.18.193.3:37969/ using document root <none> and password file <none>
I20260807 04:09:56.443629 24425 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:56.443764 24425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:56.444121 24425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:56.447355 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "e41811035a294957856ca2121dd1559f"
format_stamp: "Formatted at 2026-08-07 04:09:56 on dist-test-slave-8n49"
server_key: "b988b930c183741f11f7f551f5f4f5cd"
server_key_iv: "7967f19f28f821918e5c131a0783e20d"
server_key_version: "encryptionkey@0"
I20260807 04:09:56.448297 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "e41811035a294957856ca2121dd1559f"
format_stamp: "Formatted at 2026-08-07 04:09:56 on dist-test-slave-8n49"
server_key: "b988b930c183741f11f7f551f5f4f5cd"
server_key_iv: "7967f19f28f821918e5c131a0783e20d"
server_key_version: "encryptionkey@0"
I20260807 04:09:56.454264 24425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:09:56.457899 24440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:56.459340 24425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:09:56.459563 24425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "e41811035a294957856ca2121dd1559f"
format_stamp: "Formatted at 2026-08-07 04:09:56 on dist-test-slave-8n49"
server_key: "b988b930c183741f11f7f551f5f4f5cd"
server_key_iv: "7967f19f28f821918e5c131a0783e20d"
server_key_version: "encryptionkey@0"
I20260807 04:09:56.459822 24425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:56.466351 24425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:56.467255 24425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:56.467571 24425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:56.468783 24425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:09:56.470366 24425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:09:56.470503 24425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:56.470654 24425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:09:56.470752 24425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:56.507863 24425 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:45845
I20260807 04:09:56.507934 24557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:45845 every 8 connection(s)
I20260807 04:09:56.509326 24425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:09:56.512738 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24425
I20260807 04:09:56.512897 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:09:56.513362 19204 external_mini_cluster.cc:1428] Setting key 93a2931aeba95e353bdddf7bdfdedfe7
I20260807 04:09:56.519093 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.512225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:59055 (local address 127.18.193.62:42697)
0807 04:09:56.512487 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.512494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.514271 (+ 1777us) server_negotiation.cc:407] Connection header received
0807 04:09:56.516828 (+ 2557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.516836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.516912 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.517012 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.518381 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.518386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.518389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.518405 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.518438 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.518451 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.518547 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.518808 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.518861 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 04:09:56.520174 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.512914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:42697 (local address 127.18.193.3:59055)
0807 04:09:56.514118 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.514166 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:56.516466 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:56.517291 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:56.517300 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:56.517397 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:56.518090 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:56.518093 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:56.518131 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:56.518218 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:56.518578 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:56.518588 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:56.518595 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:09:56.518861 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 04:09:56.519084 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":406,"threads_started":1}
I20260807 04:09:56.521692 24558 heartbeater.cc:347] Connected to a master server at 127.18.193.62:42697
I20260807 04:09:56.522038 24558 heartbeater.cc:464] Registering TS with master...
I20260807 04:09:56.522625 24558 heartbeater.cc:511] Master 127.18.193.62:42697 requested a full tablet report, sending...
I20260807 04:09:56.524156 24093 ts_manager.cc:194] Registered new tserver with Master: e41811035a294957856ca2121dd1559f (127.18.193.3:45845)
I20260807 04:09:56.529227 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:09:56.536682 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.533913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.18.193.62:42697)
0807 04:09:56.534075 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.534079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.534593 (+ 514us) server_negotiation.cc:407] Connection header received
0807 04:09:56.534797 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.534801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.534862 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.534976 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.535965 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.535971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.535976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.536001 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.536033 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.536045 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.536183 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.536279 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.536341 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:09:56.540412 24093 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:09:56.547847 24093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:09:56.568454 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.560941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:41377 (local address 127.0.0.1:35782)
0807 04:09:56.561953 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.562007 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:56.562151 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:56.564329 (+ 2178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:56.564340 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:56.564455 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:56.565428 (+ 973us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:56.565434 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:56.565476 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:56.565574 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:56.567492 (+ 1918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:56.567508 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:56.567521 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 04:09:56.567718 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 04:09:56.567832 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":629,"threads_started":1}
I20260807 04:09:56.569919 24571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.561802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.18.193.2:41377)
0807 04:09:56.562472 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.563546 (+ 1074us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.563607 (+ 61us) server_negotiation.cc:407] Connection header received
0807 04:09:56.563702 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.563715 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.564000 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.565013 (+ 1013us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.567183 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.567196 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.567215 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.567238 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.567305 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.567329 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.569233 (+ 1904us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.569331 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.569461 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":1343,"threads_started":1}
I20260807 04:09:56.574375 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.562373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47806)
0807 04:09:56.562945 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.562967 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:56.563064 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:56.571805 (+ 8741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:56.571810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:56.571838 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:56.572609 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:56.572613 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:56.572630 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:56.572676 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:56.573812 (+ 1136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:56.573817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:56.573822 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:09:56.573940 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 04:09:56.573992 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":303,"threads_started":1}
I20260807 04:09:56.575174 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.562720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.18.193.1:39807)
0807 04:09:56.570818 (+ 8098us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.570830 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.570870 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:09:56.570955 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.570967 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.571256 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.572252 (+ 996us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.573554 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.573562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.573575 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.573598 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.573653 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.573672 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.574671 (+ 999us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.574746 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.574840 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7890,"thread_start_us":7741,"threads_started":1}
I20260807 04:09:56.575867 24356 tablet_service.cc:1467] Processing CreateTablet for tablet a2bd0947ab1c4cdd84d172949607fa9e (DEFAULT_TABLE table=test-table [id=359c7bb20be548bbb2429ca600605d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:56.577574 24356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2bd0947ab1c4cdd84d172949607fa9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:56.579059 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.562998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45845 (local address 127.0.0.1:53790)
0807 04:09:56.568758 (+ 5760us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.568781 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:56.568877 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:56.576657 (+ 7780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:56.576663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:56.576701 (+ 38us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:56.577327 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:56.577330 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:56.577347 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:56.577393 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:56.578648 (+ 1255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:56.578652 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:56.578658 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:09:56.578735 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 04:09:56.578786 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5627,"thread_start_us":8557,"threads_started":1}
I20260807 04:09:56.580008 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.566592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.18.193.3:45845)
0807 04:09:56.576114 (+ 9522us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.576123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.576164 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:09:56.576233 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.576240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.576431 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.577069 (+ 638us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.578409 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.578416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.578428 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.578444 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.578493 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.578512 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.579469 (+ 957us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.579534 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.579621 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9341,"thread_start_us":9162,"threads_started":1}
I20260807 04:09:56.581745 24219 tablet_service.cc:1467] Processing CreateTablet for tablet a2bd0947ab1c4cdd84d172949607fa9e (DEFAULT_TABLE table=test-table [id=359c7bb20be548bbb2429ca600605d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:56.583746 24491 tablet_service.cc:1467] Processing CreateTablet for tablet a2bd0947ab1c4cdd84d172949607fa9e (DEFAULT_TABLE table=test-table [id=359c7bb20be548bbb2429ca600605d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:09:56.584335 24219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2bd0947ab1c4cdd84d172949607fa9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:56.585747 24491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2bd0947ab1c4cdd84d172949607fa9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:56.593731 24576 tablet_bootstrap.cc:492] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Bootstrap starting.
I20260807 04:09:56.597558 24576 tablet_bootstrap.cc:654] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:56.598891 24576 log.cc:824] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:56.605559 24286 heartbeater.cc:503] Master 127.18.193.62:42697 was elected leader, sending a full tablet report...
I20260807 04:09:56.614238 24576 tablet_bootstrap.cc:492] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: No bootstrap required, opened a new log
I20260807 04:09:56.614569 24576 ts_tablet_manager.cc:1397] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.005s
I20260807 04:09:56.616605 24577 tablet_bootstrap.cc:492] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Bootstrap starting.
I20260807 04:09:56.618283 24579 tablet_bootstrap.cc:492] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18: Bootstrap starting.
I20260807 04:09:56.621855 24579 tablet_bootstrap.cc:654] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:56.621932 24577 tablet_bootstrap.cc:654] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:56.623176 24579 log.cc:824] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:56.623253 24577 log.cc:824] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:56.629352 24576 raft_consensus.cc:348] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.630033 24576 raft_consensus.cc:374] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:56.630204 24576 raft_consensus.cc:729] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd7e2b66b9b4d18bceb19b4addd45d8, State: Initialized, Role: FOLLOWER
I20260807 04:09:56.630991 24576 consensus_queue.cc:260] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.631651 24579 tablet_bootstrap.cc:492] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18: No bootstrap required, opened a new log
I20260807 04:09:56.631981 24579 ts_tablet_manager.cc:1397] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260807 04:09:56.632686 24422 heartbeater.cc:503] Master 127.18.193.62:42697 was elected leader, sending a full tablet report...
I20260807 04:09:56.633244 24576 ts_tablet_manager.cc:1428] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
I20260807 04:09:56.636282 24577 tablet_bootstrap.cc:492] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: No bootstrap required, opened a new log
I20260807 04:09:56.636626 24577 ts_tablet_manager.cc:1397] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.009s
I20260807 04:09:56.642424 24579 raft_consensus.cc:348] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.642930 24579 raft_consensus.cc:374] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:56.643046 24579 raft_consensus.cc:729] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017e9b775681494ba7d73b5875d0bc18, State: Initialized, Role: FOLLOWER
I20260807 04:09:56.643679 24579 consensus_queue.cc:260] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.645695 24579 ts_tablet_manager.cc:1428] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260807 04:09:56.648108 24577 raft_consensus.cc:348] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.648509 24577 raft_consensus.cc:374] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:56.648693 24577 raft_consensus.cc:729] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41811035a294957856ca2121dd1559f, State: Initialized, Role: FOLLOWER
I20260807 04:09:56.649041 24583 raft_consensus.cc:482] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:09:56.649329 24577 consensus_queue.cc:260] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.649384 24583 raft_consensus.cc:504] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.650600 24558 heartbeater.cc:503] Master 127.18.193.62:42697 was elected leader, sending a full tablet report...
I20260807 04:09:56.651254 24577 ts_tablet_manager.cc:1428] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260807 04:09:56.651476 24583 leader_election.cc:288] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fd7e2b66b9b4d18bceb19b4addd45d8 (127.18.193.2:41377), e41811035a294957856ca2121dd1559f (127.18.193.3:45845)
I20260807 04:09:56.655218 24571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.651816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:53311 (local address 127.18.193.2:41377)
0807 04:09:56.652034 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.652044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.652892 (+ 848us) server_negotiation.cc:407] Connection header received
0807 04:09:56.653185 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.653191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.653256 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.653328 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.654519 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.654524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.654529 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.654547 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.654586 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.654599 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.654721 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.654857 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.654923 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:09:56.655808 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.653526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:54433 (local address 127.18.193.3:45845)
0807 04:09:56.653702 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.653710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.653728 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:09:56.653776 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.653781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.653829 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.653950 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.655088 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.655094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.655099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.655118 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.655152 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.655167 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.655262 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.655496 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.655556 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:09:56.656036 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.651822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:41377 (local address 127.18.193.1:53311)
0807 04:09:56.652778 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.652803 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:56.652974 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:56.653511 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:56.653514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:56.653534 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:56.654287 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:56.654293 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:56.654316 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:56.654370 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:56.654696 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:56.654701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:56.654705 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 04:09:56.655529 (+ 824us) client_negotiation.cc:240] Negotiation successful
0807 04:09:56.655593 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":477,"threads_started":1}
I20260807 04:09:56.656494 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.652844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45845 (local address 127.18.193.1:54433)
0807 04:09:56.653462 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.653481 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:56.653603 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:56.653995 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:56.653998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:56.654014 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:09:56.654861 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:09:56.654865 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:09:56.654882 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:09:56.654923 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:09:56.655360 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:09:56.655364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:09:56.655368 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 04:09:56.656061 (+ 693us) client_negotiation.cc:240] Negotiation successful
0807 04:09:56.656118 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":313,"threads_started":1}
I20260807 04:09:56.656956 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bd0947ab1c4cdd84d172949607fa9e" candidate_uuid: "017e9b775681494ba7d73b5875d0bc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" is_pre_election: true
I20260807 04:09:56.657344 24512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bd0947ab1c4cdd84d172949607fa9e" candidate_uuid: "017e9b775681494ba7d73b5875d0bc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e41811035a294957856ca2121dd1559f" is_pre_election: true
I20260807 04:09:56.657452 24376 raft_consensus.cc:2454] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 017e9b775681494ba7d73b5875d0bc18 in term 0.
I20260807 04:09:56.657804 24512 raft_consensus.cc:2454] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 017e9b775681494ba7d73b5875d0bc18 in term 0.
I20260807 04:09:56.658157 24172 leader_election.cc:302] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017e9b775681494ba7d73b5875d0bc18, 0fd7e2b66b9b4d18bceb19b4addd45d8; no voters:
I20260807 04:09:56.658541 24583 raft_consensus.cc:2781] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:09:56.658708 24583 raft_consensus.cc:482] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:09:56.658813 24583 raft_consensus.cc:3037] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:56.661505 24583 raft_consensus.cc:504] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.662182 24583 leader_election.cc:288] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [CANDIDATE]: Term 1 election: Requested vote from peers 0fd7e2b66b9b4d18bceb19b4addd45d8 (127.18.193.2:41377), e41811035a294957856ca2121dd1559f (127.18.193.3:45845)
I20260807 04:09:56.662628 24376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bd0947ab1c4cdd84d172949607fa9e" candidate_uuid: "017e9b775681494ba7d73b5875d0bc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8"
I20260807 04:09:56.662698 24512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2bd0947ab1c4cdd84d172949607fa9e" candidate_uuid: "017e9b775681494ba7d73b5875d0bc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e41811035a294957856ca2121dd1559f"
I20260807 04:09:56.662870 24376 raft_consensus.cc:3037] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:56.662909 24512 raft_consensus.cc:3037] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:56.665436 24376 raft_consensus.cc:2454] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017e9b775681494ba7d73b5875d0bc18 in term 1.
I20260807 04:09:56.665439 24512 raft_consensus.cc:2454] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 017e9b775681494ba7d73b5875d0bc18 in term 1.
I20260807 04:09:56.665839 24172 leader_election.cc:302] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 017e9b775681494ba7d73b5875d0bc18, 0fd7e2b66b9b4d18bceb19b4addd45d8; no voters:
I20260807 04:09:56.666165 24583 raft_consensus.cc:2781] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:56.666653 24583 raft_consensus.cc:686] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [term 1 LEADER]: Becoming Leader. State: Replica: 017e9b775681494ba7d73b5875d0bc18, State: Running, Role: LEADER
I20260807 04:09:56.667129 24583 consensus_queue.cc:237] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } }
I20260807 04:09:56.671479 24093 catalog_manager.cc:5495] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 017e9b775681494ba7d73b5875d0bc18 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "017e9b775681494ba7d73b5875d0bc18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "017e9b775681494ba7d73b5875d0bc18" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 39807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 } health_report { overall_health: UNKNOWN } } }
I20260807 04:09:56.699218 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.693523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.18.193.1:39807)
0807 04:09:56.693695 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.693700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.693737 (+ 37us) server_negotiation.cc:407] Connection header received
0807 04:09:56.693944 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.693948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.694034 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.694139 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.698627 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.698633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.698636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.698655 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.698691 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.698704 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.698809 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.698925 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.698989 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:09:56.707924 24512 raft_consensus.cc:1261] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 1 FOLLOWER]: Refusing update from remote peer 017e9b775681494ba7d73b5875d0bc18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:56.707923 24376 raft_consensus.cc:1261] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 1 FOLLOWER]: Refusing update from remote peer 017e9b775681494ba7d73b5875d0bc18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:09:56.708897 24583 consensus_queue.cc:1035] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fd7e2b66b9b4d18bceb19b4addd45d8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:56.709286 24587 consensus_queue.cc:1035] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e41811035a294957856ca2121dd1559f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:09:56.735088 24093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:56.735381 24093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:09:56.739197 24093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54d4b18e1ec3415f9beb857325ace499: Sending DeleteTablet for 3 replicas of tablet a2bd0947ab1c4cdd84d172949607fa9e
I20260807 04:09:56.740934 24219 tablet_service.cc:1514] Processing DeleteTablet for tablet a2bd0947ab1c4cdd84d172949607fa9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:56 UTC) from {username='slave'} at 127.0.0.1:47806
I20260807 04:09:56.741667 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24156
I20260807 04:09:56.742158 24491 tablet_service.cc:1514] Processing DeleteTablet for tablet a2bd0947ab1c4cdd84d172949607fa9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:56 UTC) from {username='slave'} at 127.0.0.1:53790
I20260807 04:09:56.741782 24356 tablet_service.cc:1514] Processing DeleteTablet for tablet a2bd0947ab1c4cdd84d172949607fa9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:09:56 UTC) from {username='slave'} at 127.0.0.1:35782
I20260807 04:09:56.746011 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.742925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.18.193.1:39807)
0807 04:09:56.743304 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.743310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.743415 (+ 105us) server_negotiation.cc:407] Connection header received
0807 04:09:56.743710 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.743716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.743786 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.743861 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.745150 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.745159 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.745166 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.745189 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.745235 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.745254 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.745401 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.745551 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.745621 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260807 04:09:56.747159 24281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:56.748164 24602 tablet_replica.cc:330] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: stopping tablet replica
I20260807 04:09:56.748797 24602 raft_consensus.cc:2229] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:56.749468 24602 raft_consensus.cc:2258] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:56.752576 24602 ts_tablet_manager.cc:1905] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:56.761461 24604 tablet_replica.cc:330] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: stopping tablet replica
I20260807 04:09:56.762210 24604 raft_consensus.cc:2229] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:09:56.762583 24602 ts_tablet_manager.cc:1918] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:56.762817 24602 log.cc:1196] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/a2bd0947ab1c4cdd84d172949607fa9e
I20260807 04:09:56.763178 24604 raft_consensus.cc:2258] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:09:56.763504 24602 ts_tablet_manager.cc:1939] T a2bd0947ab1c4cdd84d172949607fa9e P e41811035a294957856ca2121dd1559f: Deleting consensus metadata
I20260807 04:09:56.766405 24604 ts_tablet_manager.cc:1905] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:09:56.766552 24078 catalog_manager.cc:4841] TS e41811035a294957856ca2121dd1559f (127.18.193.3:45845): tablet a2bd0947ab1c4cdd84d172949607fa9e (table test-table [id=359c7bb20be548bbb2429ca600605d7f]) successfully deleted
I20260807 04:09:56.774453 24604 ts_tablet_manager.cc:1918] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:09:56.774657 24604 log.cc:1196] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/a2bd0947ab1c4cdd84d172949607fa9e
I20260807 04:09:56.775127 24604 ts_tablet_manager.cc:1939] T a2bd0947ab1c4cdd84d172949607fa9e P 0fd7e2b66b9b4d18bceb19b4addd45d8: Deleting consensus metadata
I20260807 04:09:56.776767 24078 catalog_manager.cc:4841] TS 0fd7e2b66b9b4d18bceb19b4addd45d8 (127.18.193.2:41377): tablet a2bd0947ab1c4cdd84d172949607fa9e (table test-table [id=359c7bb20be548bbb2429ca600605d7f]) successfully deleted
I20260807 04:09:56.863134 24601 tablet_replica.cc:330] T a2bd0947ab1c4cdd84d172949607fa9e P 017e9b775681494ba7d73b5875d0bc18: stopping tablet replica
I20260807 04:09:56.863404 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24156
I20260807 04:09:56.884064 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24290
W20260807 04:09:56.884012 24078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.193.1:39807 (error 108)
I20260807 04:09:56.886004 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.885543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47840)
0807 04:09:56.885710 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.885835 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 04:09:56.886289 24078 catalog_manager.cc:4584] TS 017e9b775681494ba7d73b5875d0bc18 (127.18.193.1:39807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2bd0947ab1c4cdd84d172949607fa9e: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
I20260807 04:09:56.887468 24571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.884972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.18.193.2:41377)
0807 04:09:56.885466 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.885471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.885488 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:09:56.885539 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.885543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.885623 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:56.885742 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:56.886793 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:56.886798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:56.886801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:56.886816 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:56.886848 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:56.886860 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:56.886963 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:56.887116 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 04:09:56.887162 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380}
I20260807 04:09:56.888186 24417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:56.940156 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.939533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47852)
0807 04:09:56.939836 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.939933 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 04:09:56.941354 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.940919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47856)
0807 04:09:56.941098 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:56.941190 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 04:09:56.977873 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24290
I20260807 04:09:56.998678 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24425
I20260807 04:09:57.002363 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:56.999490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.18.193.3:45845)
0807 04:09:56.999658 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:56.999663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:56.999702 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:09:56.999868 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:56.999873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:56.999958 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:57.000046 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:57.001157 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:57.001166 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:57.001172 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:57.001194 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:57.001244 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:57.001262 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:57.001427 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:57.001946 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 04:09:57.002016 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:09:57.002903 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:57.002391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47870)
0807 04:09:57.002648 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:57.002745 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 04:09:57.003310 24553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:57.004153 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:57.003626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47878)
0807 04:09:57.003829 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:57.003939 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 04:09:57.085068 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:57.084458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47882)
0807 04:09:57.084760 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:57.084879 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 04:09:57.086225 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:57.085808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:39807 (local address 127.0.0.1:47884)
0807 04:09:57.086018 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:57.086099 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:39807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 04:09:57.103336 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24425
I20260807 04:09:57.121980 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24061
I20260807 04:09:57.123217 24124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:57.235028 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24061
2026-08-07T04:09:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2745 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 04:09:57.279956 19204 mini_oidc.cc:152] Starting JWKS server
I20260807 04:09:57.284001 19204 webserver.cc:460] Webserver started at https://127.0.0.1:40777/ using document root <none> and password file <none>
I20260807 04:09:57.284336 19204 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 04:09:57.285074 19204 webserver.cc:460] Webserver started at http://127.0.0.1:42083/ using document root <none> and password file <none>
2026-08-07T04:09:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:57Z Disabled control of system clock
I20260807 04:09:57.317803 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:46353
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:43333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:46353
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40777/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/testchainca.cert with env {}
W20260807 04:09:57.660010 24615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:57.660369 24615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:57.660539 24615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:57.667982 24615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:57.668140 24615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:57.668234 24615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:57.668349 24615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:57.668435 24615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:09:57.668536 24615 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:09:57.668635 24615 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40777/jwks.json
I20260807 04:09:57.679358 24615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:46353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:46353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40777/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:57.680788 24615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:57.682129 24615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:57.694600 24620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:57.694506 24624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:57.694645 24622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:57.695866 24615 server_base.cc:1034] running on GCE node
I20260807 04:09:57.696779 24615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:57.698284 24615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:57.699528 24615 hybrid_clock.cc:648] HybridClock initialized: now 1786075797699464 us; error 58 us; skew 500 ppm
I20260807 04:09:57.699920 24615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:09:57.702354 24615 webserver.cc:460] Webserver started at http://127.18.193.62:37639/ using document root <none> and password file <none>
I20260807 04:09:57.703073 24615 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:57.703218 24615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:57.703554 24615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:57.706645 24615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "1cabf8f7603c409a8e3b2277e87211f3"
format_stamp: "Formatted at 2026-08-07 04:09:57 on dist-test-slave-8n49"
server_key: "9c8e34de4d75bbc8fbca84c401681984"
server_key_iv: "00fc42e0a65600aa18a2cc9b526153fc"
server_key_version: "encryptionkey@0"
I20260807 04:09:57.707474 24615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "1cabf8f7603c409a8e3b2277e87211f3"
format_stamp: "Formatted at 2026-08-07 04:09:57 on dist-test-slave-8n49"
server_key: "9c8e34de4d75bbc8fbca84c401681984"
server_key_iv: "00fc42e0a65600aa18a2cc9b526153fc"
server_key_version: "encryptionkey@0"
I20260807 04:09:57.712816 24615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 04:09:57.716776 24631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:57.718235 24615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:09:57.718617 24615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "1cabf8f7603c409a8e3b2277e87211f3"
format_stamp: "Formatted at 2026-08-07 04:09:57 on dist-test-slave-8n49"
server_key: "9c8e34de4d75bbc8fbca84c401681984"
server_key_iv: "00fc42e0a65600aa18a2cc9b526153fc"
server_key_version: "encryptionkey@0"
I20260807 04:09:57.718964 24615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:57.759121 24615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:57.760066 24615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:57.760426 24615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:57.779791 24615 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:46353
I20260807 04:09:57.779811 24684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:46353 every 8 connection(s)
I20260807 04:09:57.781320 24615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:57.785065 24686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:57.789641 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24615
I20260807 04:09:57.789872 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:57.790668 19204 external_mini_cluster.cc:1428] Setting key b6a41ef4675f91e2d1e0aeee2b4233ae
I20260807 04:09:57.796908 24686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3: Bootstrap starting.
I20260807 04:09:57.800004 24686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:57.801123 24686 log.cc:824] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:57.804721 24686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3: No bootstrap required, opened a new log
I20260807 04:09:57.805199 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:57.792674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.18.193.62:46353)
0807 04:09:57.793653 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:57.793664 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:57.793708 (+ 44us) server_negotiation.cc:407] Connection header received
0807 04:09:57.795593 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:57.795615 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:57.796063 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:57.796461 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:57.797062 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:57.798372 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:57.799711 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:57.800475 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:57.800724 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:57.802049 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:57.802057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:57.802075 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:57.802865 (+ 790us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:57.802975 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:57.803008 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:57.803143 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:57.803372 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 04:09:57.803785 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":327,"threads_started":1}
I20260807 04:09:57.809662 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:09:57.814424 24686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cabf8f7603c409a8e3b2277e87211f3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46353 } }
I20260807 04:09:57.814855 24686 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:57.814981 24686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cabf8f7603c409a8e3b2277e87211f3, State: Initialized, Role: FOLLOWER
I20260807 04:09:57.815649 24686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cabf8f7603c409a8e3b2277e87211f3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46353 } }
I20260807 04:09:57.815927 24686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:57.816108 24686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:57.816344 24686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:57.819190 24686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cabf8f7603c409a8e3b2277e87211f3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46353 } }
I20260807 04:09:57.819742 24686 leader_election.cc:302] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cabf8f7603c409a8e3b2277e87211f3; no voters:
I20260807 04:09:57.820366 24686 leader_election.cc:288] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:57.820499 24691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:57.821349 24691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [term 1 LEADER]: Becoming Leader. State: Replica: 1cabf8f7603c409a8e3b2277e87211f3, State: Running, Role: LEADER
I20260807 04:09:57.821964 24691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cabf8f7603c409a8e3b2277e87211f3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46353 } }
I20260807 04:09:57.823009 24686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:57.825315 24692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cabf8f7603c409a8e3b2277e87211f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cabf8f7603c409a8e3b2277e87211f3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46353 } } }
I20260807 04:09:57.825714 24692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:57.825461 24693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cabf8f7603c409a8e3b2277e87211f3. Latest consensus state: current_term: 1 leader_uuid: "1cabf8f7603c409a8e3b2277e87211f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cabf8f7603c409a8e3b2277e87211f3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46353 } } }
I20260807 04:09:57.825914 24693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:57.829574 24696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:57.835181 24696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:57.845115 24696 catalog_manager.cc:1347] Generated new cluster ID: 0de7f7ed3d134e91b3ed420068bb49d2
I20260807 04:09:57.845254 24696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:57.863368 24696 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:57.865223 24696 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:57.890007 24696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cabf8f7603c409a8e3b2277e87211f3: Generated new TSK 0
I20260807 04:09:57.890937 24696 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 04:09:57.926008 24718 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 04:09:57.926311 24718 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 04:09:57.928670 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:57.905351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35902 (local address 127.18.193.62:46353)
0807 04:09:57.905597 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:57.905603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:57.906026 (+ 423us) server_negotiation.cc:407] Connection header received
0807 04:09:57.906432 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:57.906437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:57.906503 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:57.906626 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:09:57.907284 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:57.907983 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:57.910096 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:57.910551 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:57.910651 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:57.911261 (+ 610us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:09:57.928044 (+ 16783us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 04:09:57.928349 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35902: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 04:09:57.928815 24689 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35902: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 04:09:57.930583 24717 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:09:57.905245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:46353 (local address 127.0.0.1:35902)
0807 04:09:57.905911 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:57.905932 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:57.906118 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:57.906686 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:57.906692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:57.906707 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:09:57.907108 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:57.907120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:57.908119 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:57.908125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:57.909949 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:57.909964 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:57.910742 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:57.910749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:57.910930 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:57.911007 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:09:57.928337 (+ 17330us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 04:09:57.928758 (+ 421us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.193.62:46353: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":331,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.193.62:46353: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 04:09:57.932817 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24615
I20260807 04:09:57.934010 24680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:58.032297 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24615
2026-08-07T04:09:58Z chronyd exiting
I20260807 04:09:58.086159 19204 test_util.cc:175] -----------------------------------------------
I20260807 04:09:58.086323 19204 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786075749366013-19204-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (811 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 04:09:58.091457 19204 mini_oidc.cc:152] Starting JWKS server
I20260807 04:09:58.093508 19204 webserver.cc:460] Webserver started at https://127.0.0.1:39855/ using document root <none> and password file <none>
I20260807 04:09:58.093752 19204 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 04:09:58.094489 19204 webserver.cc:460] Webserver started at http://127.0.0.1:42847/ using document root <none> and password file <none>
2026-08-07T04:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:58Z Disabled control of system clock
I20260807 04:09:58.126526 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44471
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:44471
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39855/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/testchainca.cert with env {}
W20260807 04:09:58.459545 24725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:58.459931 24725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:58.460121 24725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:58.468530 24725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:58.468735 24725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:58.468842 24725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:58.468961 24725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:58.469059 24725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:09:58.469167 24725 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:09:58.469259 24725 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39855/jwks.json
I20260807 04:09:58.480717 24725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:44471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39855/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:58.482249 24725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:58.483754 24725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:58.497047 24734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:58.497244 24732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:58.497248 24731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:58.498328 24725 server_base.cc:1034] running on GCE node
I20260807 04:09:58.499114 24725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:58.500746 24725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:58.501950 24725 hybrid_clock.cc:648] HybridClock initialized: now 1786075798501912 us; error 37 us; skew 500 ppm
I20260807 04:09:58.502290 24725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:09:58.504765 24725 webserver.cc:460] Webserver started at http://127.18.193.62:41757/ using document root <none> and password file <none>
I20260807 04:09:58.505455 24725 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:58.505569 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:58.505898 24725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:58.508814 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "b9290fe9894d4780a55beabb42c3fd4f"
format_stamp: "Formatted at 2026-08-07 04:09:58 on dist-test-slave-8n49"
server_key: "d712b4d34f3990870dc511baf276dc8e"
server_key_iv: "fbd396809068d8de26dc31017030ea52"
server_key_version: "encryptionkey@0"
I20260807 04:09:58.509573 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "b9290fe9894d4780a55beabb42c3fd4f"
format_stamp: "Formatted at 2026-08-07 04:09:58 on dist-test-slave-8n49"
server_key: "d712b4d34f3990870dc511baf276dc8e"
server_key_iv: "fbd396809068d8de26dc31017030ea52"
server_key_version: "encryptionkey@0"
I20260807 04:09:58.514674 24725 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:09:58.518191 24740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:58.519883 24725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:09:58.520085 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "b9290fe9894d4780a55beabb42c3fd4f"
format_stamp: "Formatted at 2026-08-07 04:09:58 on dist-test-slave-8n49"
server_key: "d712b4d34f3990870dc511baf276dc8e"
server_key_iv: "fbd396809068d8de26dc31017030ea52"
server_key_version: "encryptionkey@0"
I20260807 04:09:58.520303 24725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:58.532622 24725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:58.533435 24725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:58.533740 24725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:58.552826 24725 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:44471
I20260807 04:09:58.552851 24794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:44471 every 8 connection(s)
I20260807 04:09:58.554289 24725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:58.556993 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24725
I20260807 04:09:58.557194 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:58.557879 19204 external_mini_cluster.cc:1428] Setting key fd389ef96513baad27ef3b90d85cf6a4
I20260807 04:09:58.558302 24797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:58.573740 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:58.559874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.18.193.62:44471)
0807 04:09:58.560969 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:58.560984 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:58.561049 (+ 65us) server_negotiation.cc:407] Connection header received
0807 04:09:58.563222 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:58.563250 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:58.563861 (+ 611us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:58.564318 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:58.564919 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:58.566926 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:58.568249 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:58.569309 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:58.569492 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:58.570814 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:58.570830 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:58.570849 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:58.571639 (+ 790us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:58.571741 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:58.571767 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:58.571880 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:58.572098 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 04:09:58.572431 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":358,"threads_started":1}
I20260807 04:09:58.573940 24797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f: Bootstrap starting.
I20260807 04:09:58.577293 24797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:58.577754 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:09:58.578523 24797 log.cc:824] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:58.581676 24797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f: No bootstrap required, opened a new log
I20260807 04:09:58.588950 24797 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9290fe9894d4780a55beabb42c3fd4f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44471 } }
I20260807 04:09:58.589327 24797 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:58.589425 24797 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9290fe9894d4780a55beabb42c3fd4f, State: Initialized, Role: FOLLOWER
I20260807 04:09:58.590070 24797 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9290fe9894d4780a55beabb42c3fd4f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44471 } }
I20260807 04:09:58.590320 24797 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:58.590471 24797 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:58.590649 24797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:58.592938 24797 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9290fe9894d4780a55beabb42c3fd4f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44471 } }
I20260807 04:09:58.593443 24797 leader_election.cc:302] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9290fe9894d4780a55beabb42c3fd4f; no voters:
I20260807 04:09:58.594028 24797 leader_election.cc:288] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:58.594187 24802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:58.595505 24802 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [term 1 LEADER]: Becoming Leader. State: Replica: b9290fe9894d4780a55beabb42c3fd4f, State: Running, Role: LEADER
I20260807 04:09:58.596032 24802 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9290fe9894d4780a55beabb42c3fd4f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44471 } }
I20260807 04:09:58.596724 24797 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:58.599299 24803 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9290fe9894d4780a55beabb42c3fd4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9290fe9894d4780a55beabb42c3fd4f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44471 } } }
I20260807 04:09:58.599679 24803 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:58.599416 24804 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9290fe9894d4780a55beabb42c3fd4f. Latest consensus state: current_term: 1 leader_uuid: "b9290fe9894d4780a55beabb42c3fd4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9290fe9894d4780a55beabb42c3fd4f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44471 } } }
I20260807 04:09:58.599829 24804 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:58.600741 24811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:58.606276 24811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:58.615614 24811 catalog_manager.cc:1347] Generated new cluster ID: 02cfcb8f10e841b2bdcff34e2c5225ee
I20260807 04:09:58.615718 24811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:58.648859 24811 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:58.650722 24811 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:58.664007 24811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9290fe9894d4780a55beabb42c3fd4f: Generated new TSK 0
I20260807 04:09:58.665196 24811 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 04:09:58.708508 24829 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 04:09:58.708662 24829 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 04:09:58.709674 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:58.681646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44598 (local address 127.18.193.62:44471)
0807 04:09:58.681883 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:58.681890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:58.682283 (+ 393us) server_negotiation.cc:407] Connection header received
0807 04:09:58.682565 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:58.682570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:58.682641 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:58.682776 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:09:58.683393 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:58.684178 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:58.686251 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:58.686698 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:58.686799 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:58.687400 (+ 601us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:09:58.709104 (+ 21704us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 04:09:58.709408 (+ 304us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 04:09:58.709824 24800 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 04:09:58.710057 24827 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:09:58.681469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44471 (local address 127.0.0.1:44598)
0807 04:09:58.682152 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:58.682173 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:58.682334 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:58.682822 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:58.682827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:58.682842 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:09:58.683205 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:58.683217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:58.684336 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:58.684342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:58.686090 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:58.686110 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:58.686878 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:58.686885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:58.687090 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:58.687138 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:09:58.709357 (+ 22219us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 04:09:58.709655 (+ 298us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.193.62:44471: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":338,"threads_started":1}
I20260807 04:09:58.711635 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24725
I20260807 04:09:58.712965 24789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:09:58.811585 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24725
2026-08-07T04:09:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (779 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 04:09:58.870396 19204 mini_oidc.cc:152] Starting JWKS server
I20260807 04:09:58.872323 19204 webserver.cc:460] Webserver started at https://127.0.0.1:46415/ using document root <none> and password file <none>
I20260807 04:09:58.872611 19204 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 04:09:58.873273 19204 webserver.cc:460] Webserver started at http://127.0.0.1:39911/ using document root <none> and password file <none>
2026-08-07T04:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:58Z Disabled control of system clock
I20260807 04:09:58.908034 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38861
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:35513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:38861
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46415/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 04:09:59.254413 24836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:09:59.254756 24836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:09:59.254925 24836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:09:59.262238 24836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:09:59.262349 24836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:09:59.262444 24836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:09:59.262570 24836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:09:59.262641 24836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:09:59.262737 24836 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:09:59.262794 24836 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46415/jwks.json
I20260807 04:09:59.272279 24836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:38861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46415/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:09:59.273864 24836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:09:59.276072 24836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:09:59.288471 24841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:59.288614 24842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:09:59.288511 24845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:09:59.289338 24836 server_base.cc:1034] running on GCE node
I20260807 04:09:59.290199 24836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:09:59.291630 24836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:09:59.292831 24836 hybrid_clock.cc:648] HybridClock initialized: now 1786075799292774 us; error 54 us; skew 500 ppm
I20260807 04:09:59.293211 24836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:09:59.295658 24836 webserver.cc:460] Webserver started at http://127.18.193.62:38999/ using document root <none> and password file <none>
I20260807 04:09:59.296375 24836 fs_manager.cc:359] Metadata directory not provided
I20260807 04:09:59.296489 24836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:09:59.296907 24836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:09:59.299890 24836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "bb1f74f2dc44486482f2f59171112baa"
format_stamp: "Formatted at 2026-08-07 04:09:59 on dist-test-slave-8n49"
server_key: "13c0c2dff5045061ab7f8c79294325f7"
server_key_iv: "3a6207dd550ed36083d34ff75adfbdd6"
server_key_version: "encryptionkey@0"
I20260807 04:09:59.300789 24836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "bb1f74f2dc44486482f2f59171112baa"
format_stamp: "Formatted at 2026-08-07 04:09:59 on dist-test-slave-8n49"
server_key: "13c0c2dff5045061ab7f8c79294325f7"
server_key_iv: "3a6207dd550ed36083d34ff75adfbdd6"
server_key_version: "encryptionkey@0"
I20260807 04:09:59.306193 24836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:09:59.309830 24851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:09:59.311352 24836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:09:59.311595 24836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "bb1f74f2dc44486482f2f59171112baa"
format_stamp: "Formatted at 2026-08-07 04:09:59 on dist-test-slave-8n49"
server_key: "13c0c2dff5045061ab7f8c79294325f7"
server_key_iv: "3a6207dd550ed36083d34ff75adfbdd6"
server_key_version: "encryptionkey@0"
I20260807 04:09:59.311836 24836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:09:59.324880 24836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:09:59.325739 24836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:09:59.326066 24836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:09:59.346598 24836 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:38861
I20260807 04:09:59.346621 24905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:38861 every 8 connection(s)
I20260807 04:09:59.348290 24836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:09:59.352263 24906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:09:59.352689 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24836
I20260807 04:09:59.352936 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:09:59.353670 19204 external_mini_cluster.cc:1428] Setting key 39eae8f5df2e7a4b8155a65303690fdd
I20260807 04:09:59.368031 24906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa: Bootstrap starting.
I20260807 04:09:59.370412 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:59.355848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35064 (local address 127.18.193.62:38861)
0807 04:09:59.357057 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:59.357071 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:59.357136 (+ 65us) server_negotiation.cc:407] Connection header received
0807 04:09:59.359709 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:59.359740 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:59.360422 (+ 682us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:59.360927 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:09:59.361487 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:59.363410 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:59.364932 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:59.366051 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:59.366255 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:59.367420 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:09:59.367431 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:09:59.367449 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:09:59.368193 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:09:59.368298 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:09:59.368328 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:09:59.368440 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 04:09:59.368698 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 04:09:59.369040 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":381,"threads_started":1}
I20260807 04:09:59.374684 24906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa: Neither blocks nor log segments found. Creating new log.
I20260807 04:09:59.376004 24906 log.cc:824] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa: Log is configured to *not* fsync() on all Append() calls
I20260807 04:09:59.382793 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:09:59.383064 24906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa: No bootstrap required, opened a new log
I20260807 04:09:59.390887 24906 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1f74f2dc44486482f2f59171112baa" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38861 } }
I20260807 04:09:59.391279 24906 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:09:59.391386 24906 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1f74f2dc44486482f2f59171112baa, State: Initialized, Role: FOLLOWER
I20260807 04:09:59.391988 24906 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1f74f2dc44486482f2f59171112baa" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38861 } }
I20260807 04:09:59.392234 24906 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:09:59.392396 24906 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:09:59.392584 24906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:09:59.395064 24906 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1f74f2dc44486482f2f59171112baa" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38861 } }
I20260807 04:09:59.395586 24906 leader_election.cc:302] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1f74f2dc44486482f2f59171112baa; no voters:
I20260807 04:09:59.396189 24906 leader_election.cc:288] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:09:59.396338 24911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:09:59.397135 24911 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [term 1 LEADER]: Becoming Leader. State: Replica: bb1f74f2dc44486482f2f59171112baa, State: Running, Role: LEADER
I20260807 04:09:59.397722 24911 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1f74f2dc44486482f2f59171112baa" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38861 } }
I20260807 04:09:59.398474 24906 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:09:59.400643 24913 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1f74f2dc44486482f2f59171112baa. Latest consensus state: current_term: 1 leader_uuid: "bb1f74f2dc44486482f2f59171112baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1f74f2dc44486482f2f59171112baa" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38861 } } }
I20260807 04:09:59.401046 24913 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:59.400741 24912 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb1f74f2dc44486482f2f59171112baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1f74f2dc44486482f2f59171112baa" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38861 } } }
I20260807 04:09:59.401182 24912 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa [sys.catalog]: This master's current role is: LEADER
I20260807 04:09:59.402486 24918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:09:59.408387 24918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:09:59.417678 24918 catalog_manager.cc:1347] Generated new cluster ID: e60c3885db6b42be97f4904105632b9d
I20260807 04:09:59.417809 24918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:09:59.438231 24918 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:09:59.440148 24918 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:09:59.453872 24918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb1f74f2dc44486482f2f59171112baa: Generated new TSK 0
I20260807 04:09:59.455025 24918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:09:59.478400 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:09:59.469110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.18.193.62:38861)
0807 04:09:59.469375 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 04:09:59.469385 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:09:59.469891 (+ 506us) server_negotiation.cc:407] Connection header received
0807 04:09:59.470146 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:09:59.470150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:09:59.470229 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:09:59.470371 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:09:59.470958 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:59.471877 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:59.474051 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:09:59.474543 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:09:59.474650 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:59.475277 (+ 627us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:09:59.477749 (+ 2472us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46415/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 04:09:59.478094 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46415/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 04:09:59.478531 24909 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46415/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 04:09:59.480647 24936 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:09:59.469080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38861 (local address 127.0.0.1:35066)
0807 04:09:59.469768 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 04:09:59.469788 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:09:59.469949 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:09:59.470415 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:09:59.470420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:09:59.470433 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:09:59.470784 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:59.470797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:59.472035 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:59.472043 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:59.473868 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:09:59.473884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:09:59.474750 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:09:59.474757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:09:59.474953 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:09:59.475001 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:09:59.477983 (+ 2982us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46415/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 04:09:59.480144 (+ 2161us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.193.62:38861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46415/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":1824,"thread_start_us":375,"threads_started":1}
I20260807 04:09:59.481702 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24836
I20260807 04:09:59.482861 24901 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 04:09:59.490382 24937 webserver.cc:533] Webserver: sslize failed:
E20260807 04:09:59.490478 24937 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 04:09:59.582304 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24836
2026-08-07T04:09:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (779 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T04:09:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:09:59Z Disabled control of system clock
I20260807 04:09:59.685741 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:39017
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:39017
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/testchainca.cert with env {}
W20260807 04:10:00.025215 24942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:00.025544 24942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:00.025637 24942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:00.032950 24942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:00.033068 24942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:00.033114 24942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:00.033174 24942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:00.033210 24942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:10:00.033274 24942 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:10:00.033314 24942 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 04:10:00.043411 24942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:39017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:39017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:00.044782 24942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:00.046067 24942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:00.060034 24947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:00.060045 24948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:00.060108 24950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:00.060946 24942 server_base.cc:1034] running on GCE node
I20260807 04:10:00.061898 24942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:00.063482 24942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:00.064706 24942 hybrid_clock.cc:648] HybridClock initialized: now 1786075800064662 us; error 72 us; skew 500 ppm
I20260807 04:10:00.065074 24942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:00.067507 24942 webserver.cc:460] Webserver started at http://127.18.193.62:43095/ using document root <none> and password file <none>
I20260807 04:10:00.068250 24942 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:00.068388 24942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:00.068769 24942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:00.071748 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "3f791f6771854191baed41a1b3d662cd"
format_stamp: "Formatted at 2026-08-07 04:10:00 on dist-test-slave-8n49"
server_key: "a8f6700cad1a52f7821de182294cfd82"
server_key_iv: "e8e2c9cce7179c29de165cc38a5e7678"
server_key_version: "encryptionkey@0"
I20260807 04:10:00.072710 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "3f791f6771854191baed41a1b3d662cd"
format_stamp: "Formatted at 2026-08-07 04:10:00 on dist-test-slave-8n49"
server_key: "a8f6700cad1a52f7821de182294cfd82"
server_key_iv: "e8e2c9cce7179c29de165cc38a5e7678"
server_key_version: "encryptionkey@0"
I20260807 04:10:00.078567 24942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:10:00.082226 24956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:00.083649 24942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:10:00.083849 24942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "3f791f6771854191baed41a1b3d662cd"
format_stamp: "Formatted at 2026-08-07 04:10:00 on dist-test-slave-8n49"
server_key: "a8f6700cad1a52f7821de182294cfd82"
server_key_iv: "e8e2c9cce7179c29de165cc38a5e7678"
server_key_version: "encryptionkey@0"
I20260807 04:10:00.084075 24942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:00.104990 24942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:00.105909 24942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:00.106213 24942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:00.126098 24942 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:39017
I20260807 04:10:00.126138 25009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:39017 every 8 connection(s)
I20260807 04:10:00.127458 24942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:00.128383 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 24942
I20260807 04:10:00.128643 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:00.129375 19204 external_mini_cluster.cc:1428] Setting key 82dc5a26873078dda837cba80366d7a8
I20260807 04:10:00.132408 25010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:00.142803 25013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:00.131373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.18.193.62:39017)
0807 04:10:00.132401 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:00.132416 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:00.132464 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:10:00.134170 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:00.134191 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:00.134619 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:00.135043 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:00.135619 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:00.136990 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:00.138286 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:00.138955 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:00.139163 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:00.140417 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:00.140427 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:00.140444 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:00.141074 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:00.141151 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:00.141172 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:00.141302 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:00.141553 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 04:10:00.141854 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":384,"threads_started":1}
I20260807 04:10:00.144857 25010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd: Bootstrap starting.
I20260807 04:10:00.146988 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:10:00.148452 25010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:00.149813 25010 log.cc:824] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:00.152901 25010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd: No bootstrap required, opened a new log
I20260807 04:10:00.161300 25010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f791f6771854191baed41a1b3d662cd" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39017 } }
I20260807 04:10:00.161796 25010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:00.161917 25010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f791f6771854191baed41a1b3d662cd, State: Initialized, Role: FOLLOWER
I20260807 04:10:00.162652 25010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f791f6771854191baed41a1b3d662cd" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39017 } }
I20260807 04:10:00.162890 25010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:00.163019 25010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:00.163170 25010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:00.165598 25010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f791f6771854191baed41a1b3d662cd" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39017 } }
I20260807 04:10:00.166100 25010 leader_election.cc:302] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f791f6771854191baed41a1b3d662cd; no voters:
I20260807 04:10:00.166817 25010 leader_election.cc:288] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:00.166993 25015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:00.167806 25015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [term 1 LEADER]: Becoming Leader. State: Replica: 3f791f6771854191baed41a1b3d662cd, State: Running, Role: LEADER
I20260807 04:10:00.168387 25015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f791f6771854191baed41a1b3d662cd" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39017 } }
I20260807 04:10:00.169243 25010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:00.171383 25017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f791f6771854191baed41a1b3d662cd. Latest consensus state: current_term: 1 leader_uuid: "3f791f6771854191baed41a1b3d662cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f791f6771854191baed41a1b3d662cd" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39017 } } }
I20260807 04:10:00.171854 25017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:00.172122 25016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f791f6771854191baed41a1b3d662cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f791f6771854191baed41a1b3d662cd" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39017 } } }
I20260807 04:10:00.172578 25016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:00.173880 25020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:00.178285 25020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:00.188583 25020 catalog_manager.cc:1347] Generated new cluster ID: f88f7b85f5654adc9d5890a20f428186
I20260807 04:10:00.188717 25020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:00.208052 25020 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:00.210201 25020 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:00.224629 25020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f791f6771854191baed41a1b3d662cd: Generated new TSK 0
I20260807 04:10:00.225839 25020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:10:00.324254 25013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:00.254058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.18.193.62:39017)
0807 04:10:00.254264 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:00.254271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:00.254624 (+ 353us) server_negotiation.cc:407] Connection header received
0807 04:10:00.254817 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:00.254826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:00.254956 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:00.255132 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:10:00.255593 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:00.256908 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:00.258365 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:00.259164 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:00.259313 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:00.259805 (+ 492us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:10:00.323391 (+ 63586us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 04:10:00.323919 (+ 528us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44222: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 04:10:00.324425 25013 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44222: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 04:10:00.324581 25041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:00.253921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:39017 (local address 127.0.0.1:44222)
0807 04:10:00.254514 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:00.254531 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:00.254661 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:00.255160 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:00.255164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:00.255175 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:10:00.255419 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:00.255427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:00.257074 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:00.257079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:00.258210 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:00.258221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:00.259337 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:00.259347 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:00.259485 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:00.259518 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:10:00.323784 (+ 64266us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 04:10:00.324107 (+ 323us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.193.62:39017: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":303,"threads_started":1}
I20260807 04:10:00.326007 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 24942
I20260807 04:10:00.327257 25005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:00.430167 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 24942
2026-08-07T04:10:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (818 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (436 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (432 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:10:01 dist-test-slave-8n49 krb5kdc[25056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:10:01 dist-test-slave-8n49 krb5kdc[25056](info): set up 2 sockets
Aug 07 04:10:01 dist-test-slave-8n49 krb5kdc[25056](info): commencing operation
krb5kdc: starting...
W20260807 04:10:02.454861 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:10:02 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:10:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:10:02.706645 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:39505
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:39505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:03.058395 25070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:03.058717 25070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:03.058895 25070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:03.067770 25070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:03.068001 25070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:03.068068 25070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:03.068171 25070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:03.068252 25070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:03.078720 25070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:39505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:39505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:03.080163 25070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:03.081575 25070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:03.094182 25075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:03.094149 25076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:03.094072 25078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:03.095382 25070 server_base.cc:1034] running on GCE node
I20260807 04:10:03.096246 25070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:03.097848 25070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:03.099123 25070 hybrid_clock.cc:648] HybridClock initialized: now 1786075803099075 us; error 42 us; skew 500 ppm
I20260807 04:10:03.099558 25070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:03 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075803, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:03.106416 25070 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:10:03.108237 25070 webserver.cc:460] Webserver started at http://127.18.193.62:44173/ using document root <none> and password file <none>
I20260807 04:10:03.109076 25070 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:03.109231 25070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:03.109579 25070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:03.112953 25070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "6112a3c0423d4f5caacb7db4a1c4a00f"
format_stamp: "Formatted at 2026-08-07 04:10:03 on dist-test-slave-8n49"
server_key: "beabeb8505182e92889502ab6fe2ebbc"
server_key_iv: "3d38e039e2450c162d76208dfc4d74a6"
server_key_version: "encryptionkey@0"
I20260807 04:10:03.113818 25070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "6112a3c0423d4f5caacb7db4a1c4a00f"
format_stamp: "Formatted at 2026-08-07 04:10:03 on dist-test-slave-8n49"
server_key: "beabeb8505182e92889502ab6fe2ebbc"
server_key_iv: "3d38e039e2450c162d76208dfc4d74a6"
server_key_version: "encryptionkey@0"
I20260807 04:10:03.120172 25070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 04:10:03.123854 25085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:03.125427 25070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:03.125641 25070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "6112a3c0423d4f5caacb7db4a1c4a00f"
format_stamp: "Formatted at 2026-08-07 04:10:03 on dist-test-slave-8n49"
server_key: "beabeb8505182e92889502ab6fe2ebbc"
server_key_iv: "3d38e039e2450c162d76208dfc4d74a6"
server_key_version: "encryptionkey@0"
I20260807 04:10:03.125866 25070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:03.197819 25070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:03.202618 25070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:03.202982 25070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:03.223680 25070 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:39505
I20260807 04:10:03.223737 25139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:39505 every 8 connection(s)
I20260807 04:10:03.225297 25070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:03.229425 25140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:03.234131 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 25070
I20260807 04:10:03.234360 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:03.235203 19204 external_mini_cluster.cc:1428] Setting key 9481c1af2f3204b8a2bf288145c8c196
I20260807 04:10:03.242482 25140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f: Bootstrap starting.
I20260807 04:10:03.245924 25140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:03.247031 25140 log.cc:824] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:03.250280 25140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f: No bootstrap required, opened a new log
Aug 07 04:10:03 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:03.259886 25140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39505 } }
I20260807 04:10:03.260368 25140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:03.260516 25140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6112a3c0423d4f5caacb7db4a1c4a00f, State: Initialized, Role: FOLLOWER
I20260807 04:10:03.261283 25140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39505 } }
I20260807 04:10:03.261639 25140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:03.261866 25140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:03.262099 25140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:03.264022 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:03.237499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.18.193.62:39505)
0807 04:10:03.238819 (+ 1320us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:03.238836 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:03.238903 (+ 67us) server_negotiation.cc:407] Connection header received
0807 04:10:03.241210 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:03.241238 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:03.241687 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:03.242124 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:03.243713 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:03.245216 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:03.246612 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:03.247379 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:03.247590 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:03.252151 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:03.252160 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:03.252175 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:03.252221 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:03.257641 (+ 5420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:03.258872 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:03.258885 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:03.258908 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:03.259199 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:03.259823 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:03.259828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:03.259831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:03.260770 (+ 939us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:03.261120 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:03.262467 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0807 04:10:03.262883 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1091,"thread_start_us":872,"threads_started":1}
I20260807 04:10:03.264783 25140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39505 } }
I20260807 04:10:03.265388 25140 leader_election.cc:302] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6112a3c0423d4f5caacb7db4a1c4a00f; no voters:
I20260807 04:10:03.266047 25140 leader_election.cc:288] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:03.266254 25145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:03.267553 25145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [term 1 LEADER]: Becoming Leader. State: Replica: 6112a3c0423d4f5caacb7db4a1c4a00f, State: Running, Role: LEADER
I20260807 04:10:03.268288 25145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39505 } }
I20260807 04:10:03.268438 25140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:03.272235 25148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6112a3c0423d4f5caacb7db4a1c4a00f. Latest consensus state: current_term: 1 leader_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39505 } } }
I20260807 04:10:03.272686 25148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:03.272475 25147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6112a3c0423d4f5caacb7db4a1c4a00f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 39505 } } }
I20260807 04:10:03.273093 25147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:03.274461 25154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:03.279496 25154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:03.289650 25154 catalog_manager.cc:1347] Generated new cluster ID: f24cdd6bf8a54084b8ee7adf3e167041
I20260807 04:10:03.289773 25154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:03.303727 25154 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:03.305233 25154 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:03.314438 25154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6112a3c0423d4f5caacb7db4a1c4a00f: Generated new TSK 0
I20260807 04:10:03.315474 25154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:10:03.388659 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:39505
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:03.758109 25167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:03.758512 25167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:03.758719 25167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:03.766211 25167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:03.766376 25167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:03.766561 25167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:03.777442 25167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:39505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:03.778889 25167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:03.780269 25167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:03.794212 25175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:03.794256 25173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:03.794166 25172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:03.795527 25167 server_base.cc:1034] running on GCE node
I20260807 04:10:03.796186 25167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:03.797276 25167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:03.798509 25167 hybrid_clock.cc:648] HybridClock initialized: now 1786075803798460 us; error 40 us; skew 500 ppm
I20260807 04:10:03.798854 25167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:03 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075803, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:03.802217 25167 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:10:03.803802 25167 webserver.cc:460] Webserver started at http://127.18.193.1:39647/ using document root <none> and password file <none>
I20260807 04:10:03.804495 25167 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:03.804688 25167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:03.805017 25167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:03.808003 25167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "848d6e33510d40a3964e4c039cb31ad1"
format_stamp: "Formatted at 2026-08-07 04:10:03 on dist-test-slave-8n49"
server_key: "e529b92874adbbbe166f15b5e83b4724"
server_key_iv: "5545985e821dfdd3450294d1fb8c58b3"
server_key_version: "encryptionkey@0"
I20260807 04:10:03.808910 25167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "848d6e33510d40a3964e4c039cb31ad1"
format_stamp: "Formatted at 2026-08-07 04:10:03 on dist-test-slave-8n49"
server_key: "e529b92874adbbbe166f15b5e83b4724"
server_key_iv: "5545985e821dfdd3450294d1fb8c58b3"
server_key_version: "encryptionkey@0"
I20260807 04:10:03.814210 25167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:10:03.817890 25182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:03.819362 25167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:10:03.819512 25167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "848d6e33510d40a3964e4c039cb31ad1"
format_stamp: "Formatted at 2026-08-07 04:10:03 on dist-test-slave-8n49"
server_key: "e529b92874adbbbe166f15b5e83b4724"
server_key_iv: "5545985e821dfdd3450294d1fb8c58b3"
server_key_version: "encryptionkey@0"
I20260807 04:10:03.819738 25167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:03.844416 25167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:03.849144 25167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:03.849396 25167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:03.850454 25167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:03.851850 25167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:03.851924 25167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:03.852020 25167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:03.852070 25167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:03.887925 25167 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:34441
I20260807 04:10:03.888099 25300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:34441 every 8 connection(s)
I20260807 04:10:03.889647 25167 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:03.897262 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 25167
I20260807 04:10:03.897480 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:10:03.898181 19204 external_mini_cluster.cc:1428] Setting key cf0393025e8791943c453f9fc2116d0e
Aug 07 04:10:03 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075803, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:03.911170 25303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:03.892467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:56091 (local address 127.18.193.62:39505)
0807 04:10:03.892961 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:03.892966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:03.893984 (+ 1018us) server_negotiation.cc:407] Connection header received
0807 04:10:03.895883 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:03.895889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:03.895988 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:03.896112 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:03.897810 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:03.898601 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:03.900487 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:03.900949 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:03.901099 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:03.905243 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:03.905248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:03.905252 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:03.905286 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:03.907858 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:03.908697 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:03.908701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:03.908704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:03.908836 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:03.909296 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:03.909305 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:03.909312 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:03.909592 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:03.909756 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:03.910714 (+ 958us) server_negotiation.cc:299] Negotiation successful
0807 04:10:03.910894 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260807 04:10:03.912840 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:03.893056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:39505 (local address 127.18.193.1:56091)
0807 04:10:03.893816 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:03.893856 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:03.895615 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:03.896359 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:03.896367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:03.896965 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:03.897619 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:03.897648 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:03.898755 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:03.898760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:03.900321 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:03.900335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:03.901206 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:03.901215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:03.901469 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:03.902235 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:03.902259 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:03.904996 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:03.908021 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:03.908031 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:03.908050 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:03.908507 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:03.908958 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:03.908963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:03.908967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:03.909170 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:03.909739 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:03.909747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:03.910246 (+ 499us) client_negotiation.cc:769] Sending connection context
0807 04:10:03.910561 (+ 315us) client_negotiation.cc:240] Negotiation successful
0807 04:10:03.911203 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":334,"threads_started":1}
I20260807 04:10:03.914572 25301 heartbeater.cc:347] Connected to a master server at 127.18.193.62:39505
I20260807 04:10:03.914927 25301 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:03.915758 25301 heartbeater.cc:511] Master 127.18.193.62:39505 requested a full tablet report, sending...
I20260807 04:10:03.918061 25103 ts_manager.cc:194] Registered new tserver with Master: 848d6e33510d40a3964e4c039cb31ad1 (127.18.193.1:34441)
I20260807 04:10:03.919579 25103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:56091
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:10:03.956876 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:39505
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:04.293991 25307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:04.294269 25307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:04.294368 25307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:04.301795 25307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:04.301913 25307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:04.302029 25307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:04.312067 25307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:39505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:04.313459 25307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:04.314781 25307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:04.328890 25313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:04.328855 25312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:04.329491 25315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:04.329849 25307 server_base.cc:1034] running on GCE node
I20260807 04:10:04.330497 25307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:04.331616 25307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:04.332849 25307 hybrid_clock.cc:648] HybridClock initialized: now 1786075804332792 us; error 54 us; skew 500 ppm
I20260807 04:10:04.333196 25307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:04 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075804, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:04.336790 25307 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:10:04.338281 25307 webserver.cc:460] Webserver started at http://127.18.193.2:34459/ using document root <none> and password file <none>
I20260807 04:10:04.339079 25307 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:04.339190 25307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:04.339553 25307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:04.342844 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "3a1d41db52214d809f0c38dab736df4b"
format_stamp: "Formatted at 2026-08-07 04:10:04 on dist-test-slave-8n49"
server_key: "e70babb6c377afc5d414983ccc27bc8f"
server_key_iv: "6526135cd9994aa9f8cf952aa254879b"
server_key_version: "encryptionkey@0"
I20260807 04:10:04.343655 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "3a1d41db52214d809f0c38dab736df4b"
format_stamp: "Formatted at 2026-08-07 04:10:04 on dist-test-slave-8n49"
server_key: "e70babb6c377afc5d414983ccc27bc8f"
server_key_iv: "6526135cd9994aa9f8cf952aa254879b"
server_key_version: "encryptionkey@0"
I20260807 04:10:04.349220 25307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:10:04.353011 25322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:04.354421 25307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:10:04.354651 25307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "3a1d41db52214d809f0c38dab736df4b"
format_stamp: "Formatted at 2026-08-07 04:10:04 on dist-test-slave-8n49"
server_key: "e70babb6c377afc5d414983ccc27bc8f"
server_key_iv: "6526135cd9994aa9f8cf952aa254879b"
server_key_version: "encryptionkey@0"
I20260807 04:10:04.354876 25307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:04.376345 25307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:04.381318 25307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:04.381645 25307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:04.382704 25307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:04.384150 25307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:04.384263 25307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:04.384430 25307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:04.384521 25307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:04.418339 25307 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:44715
I20260807 04:10:04.418385 25440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:44715 every 8 connection(s)
I20260807 04:10:04.419643 25307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:04.421044 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 25307
I20260807 04:10:04.421224 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:10:04.421729 19204 external_mini_cluster.cc:1428] Setting key cd21819ce95d85effe3eb216e60d96a5
Aug 07 04:10:04 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075804, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:04.444869 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:04.422356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:32893 (local address 127.18.193.62:39505)
0807 04:10:04.422821 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:04.422834 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:04.424724 (+ 1890us) server_negotiation.cc:407] Connection header received
0807 04:10:04.426979 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:04.426983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:04.427051 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:04.427151 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:04.429323 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.430203 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.432714 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.433139 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.433235 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:04.437073 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:04.437082 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:04.437090 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:04.437136 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:04.441201 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:04.442083 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:04.442091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:04.442097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:04.442275 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:04.442738 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:04.442746 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:04.442752 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:04.443142 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:04.443294 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:04.444040 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 04:10:04.444308 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260807 04:10:04.445633 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:04.422985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:39505 (local address 127.18.193.2:32893)
0807 04:10:04.423895 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:04.423932 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:04.426661 (+ 2729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:04.427440 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:04.427454 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:04.428255 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:04.429139 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:04.429167 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.430357 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.430365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:04.432573 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:04.432591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.433381 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.433387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:04.433590 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:04.434319 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:04.434346 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:04.436813 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:04.441377 (+ 4564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:04.441385 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:04.441400 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:04.441919 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:04.442390 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:04.442393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:04.442395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:04.442615 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:04.443265 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:04.443272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:04.443645 (+ 373us) client_negotiation.cc:769] Sending connection context
0807 04:10:04.443892 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 04:10:04.444306 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":339,"threads_started":1}
I20260807 04:10:04.447302 25441 heartbeater.cc:347] Connected to a master server at 127.18.193.62:39505
I20260807 04:10:04.447644 25441 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:04.448390 25441 heartbeater.cc:511] Master 127.18.193.62:39505 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
I20260807 04:10:04.449998 25103 ts_manager.cc:194] Registered new tserver with Master: 3a1d41db52214d809f0c38dab736df4b (127.18.193.2:44715)
I20260807 04:10:04.450896 25103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:32893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:10:04.483853 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:39505
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:04.816606 25447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:04.816902 25447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:04.817008 25447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:04.824434 25447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:04.824542 25447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:04.824720 25447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:04.834851 25447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:39505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:04.836170 25447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:04.837479 25447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:04.852823 25453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:04.852807 25452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:04.852896 25455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:04.854193 25447 server_base.cc:1034] running on GCE node
I20260807 04:10:04.854885 25447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:04.856148 25447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:04.857422 25447 hybrid_clock.cc:648] HybridClock initialized: now 1786075804857394 us; error 132 us; skew 500 ppm
I20260807 04:10:04.857838 25447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:04 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075804, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:04.861400 25447 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:10:04.862901 25447 webserver.cc:460] Webserver started at http://127.18.193.3:32939/ using document root <none> and password file <none>
I20260807 04:10:04.863622 25447 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:04.863726 25447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:04.863986 25447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:04.867152 25447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "68919a4f79bd4a2a83bee614f93a5cdd"
format_stamp: "Formatted at 2026-08-07 04:10:04 on dist-test-slave-8n49"
server_key: "672e8e1e873c89a9ae42faccb17c6394"
server_key_iv: "cffcfb31ab35bcb4fda23af161faf315"
server_key_version: "encryptionkey@0"
I20260807 04:10:04.867946 25447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "68919a4f79bd4a2a83bee614f93a5cdd"
format_stamp: "Formatted at 2026-08-07 04:10:04 on dist-test-slave-8n49"
server_key: "672e8e1e873c89a9ae42faccb17c6394"
server_key_iv: "cffcfb31ab35bcb4fda23af161faf315"
server_key_version: "encryptionkey@0"
I20260807 04:10:04.873570 25447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 04:10:04.877215 25463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:04.878566 25447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:10:04.878715 25447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "68919a4f79bd4a2a83bee614f93a5cdd"
format_stamp: "Formatted at 2026-08-07 04:10:04 on dist-test-slave-8n49"
server_key: "672e8e1e873c89a9ae42faccb17c6394"
server_key_iv: "cffcfb31ab35bcb4fda23af161faf315"
server_key_version: "encryptionkey@0"
I20260807 04:10:04.878902 25447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:04.906881 25447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:04.911201 25447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:04.911459 25447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:04.912458 25447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:04.913902 25447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:04.913981 25447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:04.914070 25447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:04.914117 25447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:04.923050 25301 heartbeater.cc:503] Master 127.18.193.62:39505 was elected leader, sending a full tablet report...
I20260807 04:10:04.951866 25447 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:45523
I20260807 04:10:04.951913 25583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:45523 every 8 connection(s)
I20260807 04:10:04.953313 25447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:04.957816 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 25447
I20260807 04:10:04.958000 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:10:04.958523 19204 external_mini_cluster.cc:1428] Setting key 4d04a434ad16a3838468d0e69b5649be
Aug 07 04:10:04 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075804, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:04.974701 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:04.955944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:48423 (local address 127.18.193.62:39505)
0807 04:10:04.956428 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:04.956433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:04.957487 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 04:10:04.959833 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:04.959838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:04.959937 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:04.960073 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:04.962160 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.962925 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.964538 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.964992 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.965138 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:04.968823 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:04.968828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:04.968831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:04.968863 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:04.971321 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:04.972078 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:04.972082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:04.972084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:04.972204 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:04.972732 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:04.972743 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:04.972745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:04.973026 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:04.973205 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:04.974037 (+ 832us) server_negotiation.cc:299] Negotiation successful
0807 04:10:04.974286 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":226,"threads_started":1}
I20260807 04:10:04.975759 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:04.956599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:39505 (local address 127.18.193.3:48423)
0807 04:10:04.957314 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:04.957358 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:04.959552 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:04.960331 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:04.960339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:04.961251 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:04.961988 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:04.962006 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.963078 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.963084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:04.964386 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:04.964399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:04.965245 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:04.965253 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:04.965481 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:04.966200 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:04.966223 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:04.968511 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:04.971470 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:04.971480 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:04.971498 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:04.971928 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:04.972318 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:04.972323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:04.972327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:04.972545 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:04.973170 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:04.973179 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:04.973615 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 04:10:04.973889 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 04:10:04.974512 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":288,"threads_started":1}
I20260807 04:10:04.977281 25584 heartbeater.cc:347] Connected to a master server at 127.18.193.62:39505
I20260807 04:10:04.977658 25584 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:04.978442 25584 heartbeater.cc:511] Master 127.18.193.62:39505 requested a full tablet report, sending...
I20260807 04:10:04.980304 25103 ts_manager.cc:194] Registered new tserver with Master: 68919a4f79bd4a2a83bee614f93a5cdd (127.18.193.3:45523)
I20260807 04:10:04.981179 25103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:48423
I20260807 04:10:04.987146 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/test-admin.keytab.
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:05.020303 19204 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 04:10:05.025854 25592 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:05.023137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58226 (local address 127.18.193.1:34441)
0807 04:10:05.023873 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.023880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.023911 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:10:05.023974 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.023981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.024203 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.024333 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.025431 (+ 1098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58226: BlockingRecv error: recv got EOF from 127.0.0.1:58226 (error 108)
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":388,"threads_started":1}
W20260807 04:10:05.025986 19204 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5cc' of type FILE
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:05.027871 19204 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:05.029311 19204 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:10:05.045382 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:05.031191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58228 (local address 127.18.193.1:34441)
0807 04:10:05.031397 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.031403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.031573 (+ 170us) server_negotiation.cc:407] Connection header received
0807 04:10:05.031716 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.031719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.031776 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.031858 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.032876 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.033892 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.035136 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.035691 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.035787 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:05.039468 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:05.039478 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:05.039494 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:05.039527 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:05.042551 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.043315 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.043321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.043324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.043444 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.043770 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.043773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.043776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.044143 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:05.044356 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:05.044781 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 04:10:05.045044 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 04:10:05.051928 25592 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:05.050090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.18.193.1:34441)
0807 04:10:05.050318 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.050327 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.050474 (+ 147us) server_negotiation.cc:407] Connection header received
0807 04:10:05.050631 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.050634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.050696 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.050777 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.051616 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58244: BlockingRecv error: recv got EOF from 127.0.0.1:58244 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260807 04:10:05.052008 19204 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5cc' of type FILE
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:05.053690 19204 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:05.055136 19204 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:10:05.071048 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:05.056769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.18.193.1:34441)
0807 04:10:05.056987 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.056995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.057205 (+ 210us) server_negotiation.cc:407] Connection header received
0807 04:10:05.057334 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.057338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.057404 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.057530 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.058625 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.059367 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.060892 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.061491 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.061649 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:05.065841 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:05.065847 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:05.065850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:05.065914 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:05.068427 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.069233 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.069237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.069239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.069363 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.069792 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.069796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.069798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.070136 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:05.070371 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:05.070649 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 04:10:05.070815 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 04:10:05.076930 25592 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:05.074702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.18.193.1:34441)
0807 04:10:05.074858 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.074863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.075487 (+ 624us) server_negotiation.cc:407] Connection header received
0807 04:10:05.075672 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.075677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.075761 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.075864 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.076661 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58260: BlockingRecv error: recv got EOF from 127.0.0.1:58260 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 04:10:05.077085 19204 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786075749366013-19204-0/krb5kdc/krb5cc' of type FILE
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:05.078665 19204 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:05.080009 19204 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:10:05.094856 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:05.081308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.18.193.1:34441)
0807 04:10:05.081469 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.081473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.081860 (+ 387us) server_negotiation.cc:407] Connection header received
0807 04:10:05.082077 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.082082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.082152 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.082268 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.083194 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.084053 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.085349 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.085801 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.085898 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:05.089398 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:05.089403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:05.089406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:05.089439 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:05.092096 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.092844 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.092851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.092855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.092992 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.093391 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.093395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.093397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.093684 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:05.093968 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:05.094309 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 04:10:05.094521 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:10:05.096503 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 25167
I20260807 04:10:05.107574 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:05.097260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.18.193.1:34441)
0807 04:10:05.097423 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.097428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.097775 (+ 347us) server_negotiation.cc:407] Connection header received
0807 04:10:05.097894 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.097897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.097950 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.098030 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.098875 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.099525 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.100704 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.101126 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.101221 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:05.103101 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:05.103106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:05.103109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:05.103137 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:05.105502 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.106066 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.106070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.106072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.106209 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.106495 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.106498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.106499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.106727 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:05.106918 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:05.107173 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 04:10:05.107322 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:10:05.108193 25296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:05.193135 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 25167
I20260807 04:10:05.211517 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 25307
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:10:05.227378 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:05.212511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.18.193.2:44715)
0807 04:10:05.213149 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.213156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.213183 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:10:05.213255 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.213262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.213482 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.213619 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.214589 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.215669 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.216945 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.217510 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.217613 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:05.221407 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:05.221420 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:05.221436 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:05.221478 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:05.224466 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.225308 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.225315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.225317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.225442 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.225836 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.225839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.225841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.226186 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:05.226420 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:05.226751 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 04:10:05.226960 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":342,"threads_started":1}
I20260807 04:10:05.228978 25436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:05.317848 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 25307
I20260807 04:10:05.336073 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 25447
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:10:05.353708 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:05.337659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.18.193.3:45523)
0807 04:10:05.338434 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:05.338441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:05.338473 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:10:05.338534 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:05.338545 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:05.338754 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:05.338883 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:05.340063 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.341438 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.342735 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:05.343382 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:05.343539 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:05.347754 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:05.347766 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:05.347781 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:05.347818 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:05.350767 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.351523 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.351530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.351532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.351652 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:05.352047 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:05.352051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:05.352054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:05.352434 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:05.352786 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:05.353105 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 04:10:05.353367 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":473,"threads_started":1}
I20260807 04:10:05.355120 25578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:05.442814 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 25447
I20260807 04:10:05.460891 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 25070
I20260807 04:10:05.462119 25135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:05.569242 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 25070
2026-08-07T04:10:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4275 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25612](info): set up 2 sockets
Aug 07 04:10:05 dist-test-slave-8n49 krb5kdc[25612](info): commencing operation
krb5kdc: starting...
W20260807 04:10:06.730427 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:10:06 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:10:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab.
Entry for principal oryx/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab.
I20260807 04:10:06.988461 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35583
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab
--principal=oryx/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:35583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:07.340267 25625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:07.340662 25625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:07.340785 25625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:07.348260 25625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:07.348373 25625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:07.348428 25625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:07.348482 25625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:07.348527 25625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:07.358847 25625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:35583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab
--principal=oryx/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:07.360204 25625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:07.361577 25625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:07.375891 25631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:07.375856 25633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:07.375942 25630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:07.376907 25625 server_base.cc:1034] running on GCE node
I20260807 04:10:07.377971 25625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:07.379914 25625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:07.381210 25625 hybrid_clock.cc:648] HybridClock initialized: now 1786075807381138 us; error 43 us; skew 500 ppm
I20260807 04:10:07.381649 25625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:07 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075807, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:07.385226 25625 init.cc:377] Logged in from keytab as oryx/127.18.193.62@KRBTEST.COM (short username oryx)
I20260807 04:10:07.386958 25625 webserver.cc:460] Webserver started at http://127.18.193.62:42961/ using document root <none> and password file <none>
I20260807 04:10:07.387684 25625 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:07.387836 25625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:07.388185 25625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:07.391322 25625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8"
format_stamp: "Formatted at 2026-08-07 04:10:07 on dist-test-slave-8n49"
server_key: "9ada97dca68d43f4a3a8c39393667b34"
server_key_iv: "36ca4ffbe99b2c4aabf289a8e7b91287"
server_key_version: "encryptionkey@0"
I20260807 04:10:07.392128 25625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8"
format_stamp: "Formatted at 2026-08-07 04:10:07 on dist-test-slave-8n49"
server_key: "9ada97dca68d43f4a3a8c39393667b34"
server_key_iv: "36ca4ffbe99b2c4aabf289a8e7b91287"
server_key_version: "encryptionkey@0"
I20260807 04:10:07.397682 25625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 04:10:07.401204 25641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:07.402617 25625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:10:07.402837 25625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8"
format_stamp: "Formatted at 2026-08-07 04:10:07 on dist-test-slave-8n49"
server_key: "9ada97dca68d43f4a3a8c39393667b34"
server_key_iv: "36ca4ffbe99b2c4aabf289a8e7b91287"
server_key_version: "encryptionkey@0"
I20260807 04:10:07.403061 25625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:07.420100 25625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:07.424366 25625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:07.424727 25625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:07.444726 25625 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:35583
I20260807 04:10:07.444761 25697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:35583 every 8 connection(s)
I20260807 04:10:07.446178 25625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:07.450281 25699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:07.453922 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 25625
I20260807 04:10:07.454155 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:07.454996 19204 external_mini_cluster.cc:1428] Setting key b0f0bdf68ca769de8982e9b9b94c511e
I20260807 04:10:07.464779 25699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: Bootstrap starting.
I20260807 04:10:07.469211 25699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:07.470686 25699 log.cc:824] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:10:07 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:07.474327 25699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: No bootstrap required, opened a new log
I20260807 04:10:07.479795 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:07.457231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.18.193.62:35583)
0807 04:10:07.458053 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:07.458065 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:07.458108 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:10:07.459834 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:07.459862 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:07.460282 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:07.460726 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:07.462723 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:07.464022 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:07.465948 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:07.466627 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:07.466763 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:07.472104 (+ 5341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:07.472114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:07.472129 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:07.472167 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:07.475530 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:07.476275 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:07.476281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:07.476287 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:07.476460 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:07.476904 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:07.476908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:07.476910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:07.477467 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:07.477808 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:07.478287 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 04:10:07.478721 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":394,"threads_started":1}
I20260807 04:10:07.483459 25699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } }
I20260807 04:10:07.483875 25699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:07.484010 25699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3b1030fdf24283adc4bb0f87f8dfc8, State: Initialized, Role: FOLLOWER
I20260807 04:10:07.484700 25699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } }
I20260807 04:10:07.484956 25699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:07.485133 25699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:07.485311 25699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:07.487761 25699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } }
I20260807 04:10:07.488315 25699 leader_election.cc:302] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d3b1030fdf24283adc4bb0f87f8dfc8; no voters:
I20260807 04:10:07.489065 25699 leader_election.cc:288] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:07.489315 25704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:07.490067 25704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 1 LEADER]: Becoming Leader. State: Replica: 0d3b1030fdf24283adc4bb0f87f8dfc8, State: Running, Role: LEADER
I20260807 04:10:07.490666 25704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } }
I20260807 04:10:07.491626 25699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:07.493768 25705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } }
I20260807 04:10:07.494374 25705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:07.494539 25706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d3b1030fdf24283adc4bb0f87f8dfc8. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } }
I20260807 04:10:07.494890 25706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:07.497117 25713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:07.501848 25713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:07.511518 25713 catalog_manager.cc:1347] Generated new cluster ID: e74ee0415bb5441bbdd3cc3eccdb9151
I20260807 04:10:07.511678 25713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:07.539024 25713 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:07.541191 25713 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:07.552042 25713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: Generated new TSK 0
I20260807 04:10:07.553319 25713 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab.
Entry for principal oryx/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab.
I20260807 04:10:07.616464 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35583
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab
--principal=oryx/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:07.964088 25726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:07.964393 25726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:07.964491 25726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:07.972096 25726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:07.972211 25726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:07.972419 25726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:07.982944 25726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab
--principal=oryx/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:07.984321 25726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:07.985723 25726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:07.999856 25735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:07.999908 25732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:07.999943 25731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:08.001190 25726 server_base.cc:1034] running on GCE node
I20260807 04:10:08.001845 25726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:08.002782 25726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:08.003986 25726 hybrid_clock.cc:648] HybridClock initialized: now 1786075808003947 us; error 31 us; skew 500 ppm
I20260807 04:10:08.004310 25726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:08 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:08.007578 25726 init.cc:377] Logged in from keytab as oryx/127.18.193.1@KRBTEST.COM (short username oryx)
I20260807 04:10:08.009328 25726 webserver.cc:460] Webserver started at http://127.18.193.1:37989/ using document root <none> and password file <none>
I20260807 04:10:08.010038 25726 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:08.010181 25726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:08.010519 25726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:08.013829 25726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "64c3784d514fc88f62c56d7c1b408e23"
server_key_iv: "4d96b6379f983791511df55728da221a"
server_key_version: "encryptionkey@0"
I20260807 04:10:08.014703 25726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "64c3784d514fc88f62c56d7c1b408e23"
server_key_iv: "4d96b6379f983791511df55728da221a"
server_key_version: "encryptionkey@0"
I20260807 04:10:08.020253 25726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:10:08.024014 25743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:08.025620 25726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:08.025826 25726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "64c3784d514fc88f62c56d7c1b408e23"
server_key_iv: "4d96b6379f983791511df55728da221a"
server_key_version: "encryptionkey@0"
I20260807 04:10:08.026059 25726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:08.053411 25726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:08.058342 25726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:08.058682 25726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:08.059733 25726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:08.061220 25726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:08.061297 25726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:08.061467 25726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:08.061532 25726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:08.096719 25726 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:45067
I20260807 04:10:08.096805 25857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:45067 every 8 connection(s)
I20260807 04:10:08.098124 25726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:08.102988 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 25726
I20260807 04:10:08.103222 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:10:08.104249 19204 external_mini_cluster.cc:1428] Setting key 4ee952677b65e2a548ef4756316aa409
Aug 07 04:10:08 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:08.130277 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:08.100757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:36033 (local address 127.18.193.62:35583)
0807 04:10:08.101225 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:08.101230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:08.102315 (+ 1085us) server_negotiation.cc:407] Connection header received
0807 04:10:08.105345 (+ 3030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:08.105351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:08.105597 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:08.105771 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:08.108306 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.109162 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.112093 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.112542 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.112743 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:08.121697 (+ 8954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:08.121712 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:08.121721 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:08.121783 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:08.126050 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:08.126990 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:08.127000 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:08.127007 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:08.127187 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:08.127745 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:08.127754 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:08.127766 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:08.128231 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:08.128444 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:08.129347 (+ 903us) server_negotiation.cc:299] Negotiation successful
0807 04:10:08.129707 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":361,"thread_start_us":252,"threads_started":1}
I20260807 04:10:08.130985 25862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:08.101403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:36033)
0807 04:10:08.102156 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:08.102193 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:08.105014 (+ 2821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:08.106128 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:08.106138 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:08.107040 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:08.108061 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:08.108097 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.109357 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.109366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:08.111851 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:08.111874 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.112878 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.112889 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:08.113313 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:08.118410 (+ 5097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:08.118437 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:08.121359 (+ 2922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:08.126263 (+ 4904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:08.126272 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:08.126289 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:08.126831 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:08.127372 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:08.127376 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:08.127379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:08.127603 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:08.128447 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:08.128457 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:08.128903 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 04:10:08.129156 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 04:10:08.129698 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":280,"threads_started":1}
I20260807 04:10:08.132906 25858 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35583
I20260807 04:10:08.133251 25858 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.18.193.2@KRBTEST.COM; defaulting to no policy
I20260807 04:10:08.134053 25858 heartbeater.cc:511] Master 127.18.193.62:35583 requested a full tablet report, sending...
I20260807 04:10:08.136264 25660 ts_manager.cc:194] Registered new tserver with Master: 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.2@KRBTEST.COM" created.
I20260807 04:10:08.138530 25660 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:36033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab.
Entry for principal oryx/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab.
I20260807 04:10:08.169086 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35583
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab
--principal=oryx/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:08.521579 25865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:08.521930 25865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:08.522118 25865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:08.529732 25865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:08.529901 25865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:08.530120 25865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:08.539995 25865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab
--principal=oryx/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:08.541378 25865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:08.542757 25865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:08.556413 25872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:08.556378 25876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:08.556439 25874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:08.557463 25865 server_base.cc:1034] running on GCE node
I20260807 04:10:08.558125 25865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:08.559172 25865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:08.560443 25865 hybrid_clock.cc:648] HybridClock initialized: now 1786075808560368 us; error 43 us; skew 500 ppm
I20260807 04:10:08.560864 25865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:08 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:08.564457 25865 init.cc:377] Logged in from keytab as oryx/127.18.193.2@KRBTEST.COM (short username oryx)
I20260807 04:10:08.566605 25865 webserver.cc:460] Webserver started at http://127.18.193.2:36187/ using document root <none> and password file <none>
I20260807 04:10:08.567288 25865 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:08.567395 25865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:08.567684 25865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:08.571022 25865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "474dd01821e746dfbdcf7fd31daaa13a"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "eb4a872f29e3b0d19dacedf8c4219d93"
server_key_iv: "181fef265f3b57096628e67d64deef78"
server_key_version: "encryptionkey@0"
I20260807 04:10:08.571794 25865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "474dd01821e746dfbdcf7fd31daaa13a"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "eb4a872f29e3b0d19dacedf8c4219d93"
server_key_iv: "181fef265f3b57096628e67d64deef78"
server_key_version: "encryptionkey@0"
I20260807 04:10:08.577278 25865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:10:08.580972 25883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:08.582409 25865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:10:08.582635 25865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "474dd01821e746dfbdcf7fd31daaa13a"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "eb4a872f29e3b0d19dacedf8c4219d93"
server_key_iv: "181fef265f3b57096628e67d64deef78"
server_key_version: "encryptionkey@0"
I20260807 04:10:08.582849 25865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:08.603092 25865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:08.607414 25865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:08.607798 25865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:08.608880 25865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:08.610316 25865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:08.610420 25865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:08.610592 25865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:08.610669 25865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:08.646600 25865 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:33879
I20260807 04:10:08.646694 26000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:33879 every 8 connection(s)
I20260807 04:10:08.647954 25865 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:08.654717 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 25865
I20260807 04:10:08.654958 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:10:08.655443 19204 external_mini_cluster.cc:1428] Setting key c160ad0503c99afbb786c7d2ee0bb7b9
Aug 07 04:10:08 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:08.671571 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:08.650800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:48637 (local address 127.18.193.62:35583)
0807 04:10:08.651284 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:08.651289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:08.652291 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 04:10:08.654318 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:08.654324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:08.654419 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:08.654549 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:08.656275 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.657482 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.659086 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.659534 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.659685 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:08.663635 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:08.663641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:08.663644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:08.663679 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:08.666542 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:08.667621 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:08.667626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:08.667628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:08.667781 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:08.668337 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:08.668341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:08.668342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:08.668717 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:08.668841 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:08.669969 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0807 04:10:08.671272 (+ 1303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":243,"threads_started":1}
I20260807 04:10:08.672171 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:08.651368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.2:48637)
0807 04:10:08.652109 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:08.652157 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:08.654008 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:08.654809 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:08.654819 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:08.655438 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:08.656090 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:08.656110 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.657649 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.657654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:08.658926 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:08.658939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:08.659791 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:08.659799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:08.660043 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:08.660889 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:08.660919 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:08.663365 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:08.666730 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:08.666741 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:08.666761 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:08.667475 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:08.667917 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:08.667926 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:08.667933 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:08.668210 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:08.668855 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:08.668867 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:08.669394 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 04:10:08.669789 (+ 395us) client_negotiation.cc:240] Negotiation successful
0807 04:10:08.670500 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":293,"threads_started":1}
I20260807 04:10:08.673992 26001 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35583
I20260807 04:10:08.674357 26001 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:08.675366 26001 heartbeater.cc:511] Master 127.18.193.62:35583 requested a full tablet report, sending...
I20260807 04:10:08.677367 25660 ts_manager.cc:194] Registered new tserver with Master: 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879)
I20260807 04:10:08.678815 25660 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:48637
WARNING: no policy specified for oryx/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab.
Entry for principal oryx/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab.
I20260807 04:10:08.716815 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35583
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab
--principal=oryx/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:09.068341 26007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:09.068751 26007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:09.068926 26007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:09.076749 26007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:09.076858 26007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:09.077071 26007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:09.088842 26007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab
--principal=oryx/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:09.090523 26007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:09.091969 26007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:09.107121 26012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:09.107169 26016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:09.107084 26013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:09.110095 26007 server_base.cc:1034] running on GCE node
I20260807 04:10:09.110744 26007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:09.111759 26007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:09.113000 26007 hybrid_clock.cc:648] HybridClock initialized: now 1786075809112924 us; error 69 us; skew 500 ppm
I20260807 04:10:09.113405 26007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:09 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:09.116854 26007 init.cc:377] Logged in from keytab as oryx/127.18.193.3@KRBTEST.COM (short username oryx)
I20260807 04:10:09.118422 26007 webserver.cc:460] Webserver started at http://127.18.193.3:35783/ using document root <none> and password file <none>
I20260807 04:10:09.119128 26007 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:09.119230 26007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:09.119494 26007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:09.122550 26007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "e6a63168405b4888b2ce768da03f0709"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "8feddf41cbfa5fd888f11bcae8a67a10"
server_key_iv: "01a5b7a629f1c2754a807e60cedd23fc"
server_key_version: "encryptionkey@0"
I20260807 04:10:09.123298 26007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "e6a63168405b4888b2ce768da03f0709"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "8feddf41cbfa5fd888f11bcae8a67a10"
server_key_iv: "01a5b7a629f1c2754a807e60cedd23fc"
server_key_version: "encryptionkey@0"
I20260807 04:10:09.128494 26007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:10:09.132196 26024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:09.133663 26007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:09.133869 26007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "e6a63168405b4888b2ce768da03f0709"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "8feddf41cbfa5fd888f11bcae8a67a10"
server_key_iv: "01a5b7a629f1c2754a807e60cedd23fc"
server_key_version: "encryptionkey@0"
I20260807 04:10:09.134106 26007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:09.141954 25858 heartbeater.cc:503] Master 127.18.193.62:35583 was elected leader, sending a full tablet report...
I20260807 04:10:09.157908 26007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:09.163288 26007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:09.163606 26007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:09.164644 26007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:09.166077 26007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:09.166157 26007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:09.166245 26007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:09.166291 26007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:09.202566 26007 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:37081
I20260807 04:10:09.202662 26140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:37081 every 8 connection(s)
I20260807 04:10:09.203986 26007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:09.213351 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 26007
I20260807 04:10:09.213599 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:10:09.214202 19204 external_mini_cluster.cc:1428] Setting key a5c7f56be1d075f2a2db31e0c28c503a
Aug 07 04:10:09 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:09.225252 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.206762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:40239 (local address 127.18.193.62:35583)
0807 04:10:09.207337 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:09.207345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:09.208410 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 04:10:09.210365 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:09.210370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:09.210450 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:09.210554 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:09.212081 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.213036 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.215257 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.215721 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.215864 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.219523 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:09.219528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:09.219532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:09.219566 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:09.222172 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.222892 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.222898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.222902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.223030 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.223456 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.223463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.223466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.223754 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:09.223911 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:09.224645 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 04:10:09.224865 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":288,"threads_started":1}
I20260807 04:10:09.226188 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.207428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.3:40239)
0807 04:10:09.208229 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:09.208289 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:09.210087 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:09.210758 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:09.210766 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:09.211313 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:09.211900 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.211917 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.213201 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.213207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.215092 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.215105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.215943 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.215948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.216224 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.216993 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:09.217016 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:09.219305 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:09.222318 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.222327 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.222343 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.222767 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.223141 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.223147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.223150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.223335 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.223892 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:09.223900 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:09.224235 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 04:10:09.224503 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 04:10:09.225071 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":269,"threads_started":1}
I20260807 04:10:09.227578 26141 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35583
I20260807 04:10:09.227917 26141 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:09.228839 26141 heartbeater.cc:511] Master 127.18.193.62:35583 requested a full tablet report, sending...
I20260807 04:10:09.230460 25660 ts_manager.cc:194] Registered new tserver with Master: e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081)
I20260807 04:10:09.231357 25660 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:40239
I20260807 04:10:09.242332 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.193.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.61.keytab.
Entry for principal oryx/127.18.193.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.61.keytab.
I20260807 04:10:09.301196 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:46661
--webserver_interface=127.18.193.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.61.keytab
--principal=oryx/127.18.193.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:35583,127.18.193.61:46661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:09.651678 26148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:09.652032 26148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:09.652207 26148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:09.660046 26148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:09.660214 26148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:09.660337 26148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:09.660437 26148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:09.660533 26148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:09.671101 26148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:35583,127.18.193.61:46661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.61.keytab
--principal=oryx/127.18.193.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:46661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.193.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
W20260807 04:10:09.671571 26148 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.193.62:35583,127.18.193.61:46661'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 04:10:09.672909 26148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:09.674322 26148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 04:10:09.682624 26001 heartbeater.cc:503] Master 127.18.193.62:35583 was elected leader, sending a full tablet report...
W20260807 04:10:09.686961 26154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:09.686992 26157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:09.687345 26155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:09.688532 26148 server_base.cc:1034] running on GCE node
I20260807 04:10:09.689445 26148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:09.691098 26148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:09.692385 26148 hybrid_clock.cc:648] HybridClock initialized: now 1786075809692282 us; error 68 us; skew 500 ppm
I20260807 04:10:09.692787 26148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:09 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:09.696056 26148 init.cc:377] Logged in from keytab as oryx/127.18.193.61@KRBTEST.COM (short username oryx)
I20260807 04:10:09.697639 26148 webserver.cc:460] Webserver started at http://127.18.193.61:37433/ using document root <none> and password file <none>
I20260807 04:10:09.698334 26148 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:09.698429 26148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:09.698707 26148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:09.701838 26148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/instance:
uuid: "355f9a504f564ffd82691adf27dab598"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "1c1465b5973525a5747c264b2f55770d"
server_key_iv: "f9cbe8e4e8645b1eb93786b9e481f669"
server_key_version: "encryptionkey@0"
I20260807 04:10:09.702631 26148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal/instance:
uuid: "355f9a504f564ffd82691adf27dab598"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "1c1465b5973525a5747c264b2f55770d"
server_key_iv: "f9cbe8e4e8645b1eb93786b9e481f669"
server_key_version: "encryptionkey@0"
I20260807 04:10:09.708680 26148 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.007s
I20260807 04:10:09.712826 26165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:09.714481 26148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:10:09.714797 26148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
uuid: "355f9a504f564ffd82691adf27dab598"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "1c1465b5973525a5747c264b2f55770d"
server_key_iv: "f9cbe8e4e8645b1eb93786b9e481f669"
server_key_version: "encryptionkey@0"
I20260807 04:10:09.715054 26148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:09.736039 26148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:09.740986 26148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:09.741289 26148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:09.762036 26148 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.61:46661
I20260807 04:10:09.762094 26218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.61:46661 every 8 connection(s)
I20260807 04:10:09.763511 26148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
I20260807 04:10:09.766593 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 26148
I20260807 04:10:09.767891 26219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:09.778514 26219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:10:09 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.61@KRBTEST.COM
Aug 07 04:10:09 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.61@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:09.794649 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.769234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.18.193.61:46661)
0807 04:10:09.770487 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:09.770514 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:09.770571 (+ 57us) server_negotiation.cc:407] Connection header received
0807 04:10:09.773111 (+ 2540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:09.773142 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:09.773756 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:09.774250 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:09.775680 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.777734 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.779111 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.780139 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.780319 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.785777 (+ 5458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:09.785790 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:09.785807 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:09.785862 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:09.789579 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.790417 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.790422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.790429 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.790603 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.791075 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.791080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.791084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.791830 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:09.792116 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:09.792716 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 04:10:09.793176 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1001,"thread_start_us":515,"threads_started":1}
I20260807 04:10:09.799013 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.781433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51234)
0807 04:10:09.782422 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:09.782496 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:09.782703 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:09.783352 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:09.783358 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:09.784229 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:09.784658 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.784675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.785894 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.785900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.787575 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.787589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.788675 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.788681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.788818 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.789463 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:09.789493 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:09.792396 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:09.796221 (+ 3825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.796230 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.796237 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.796907 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.797352 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.797357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.797360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.797514 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.798159 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:09.798165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:09.798319 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 04:10:09.798494 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 04:10:09.798739 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":387,"threads_started":1}
I20260807 04:10:09.799036 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.781499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.18.193.62:35583)
0807 04:10:09.782943 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:09.782950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:09.782972 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:10:09.783043 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:09.783049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:09.783152 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:09.783288 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:09.784861 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.785674 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.787714 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.788354 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.788508 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.792651 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:09.792657 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:09.792662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:09.792726 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:09.796073 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.797055 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.797062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.797068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.797214 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.797654 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.797658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.797660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.798023 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:09.798182 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:09.798616 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 04:10:09.798796 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300,"thread_start_us":1199,"threads_started":1}
I20260807 04:10:09.800879 26219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:10:09 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.61@KRBTEST.COM for oryx/127.18.193.61@KRBTEST.COM
I20260807 04:10:09.814770 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.801750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39866)
0807 04:10:09.801939 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:09.801954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:09.802036 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:09.802298 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:09.802301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:09.802622 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:09.802762 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.802770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.804285 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.804294 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.805531 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.805545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.806382 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.806387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.806596 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.807210 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:09.807228 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:09.809579 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:09.812504 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.812508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.812511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.812979 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.813338 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.813342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.813344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.813477 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.813998 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:09.814002 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:09.814194 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 04:10:09.814302 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 04:10:09.814529 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 04:10:09.814944 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.801777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.18.193.61:46661)
0807 04:10:09.801990 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:09.801995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:09.802011 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:10:09.802130 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:09.802133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:09.802186 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:09.802262 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:09.802928 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.804101 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.805733 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.806224 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.806342 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.809752 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:09.809757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:09.809761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:09.809790 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:09.812351 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.813087 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.813091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.813093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.813214 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.813599 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.813602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.813604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.813885 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:09.814003 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:09.814398 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 04:10:09.814656 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":86}
I20260807 04:10:09.819948 26219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Bootstrap starting.
I20260807 04:10:09.823155 26219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:09.824231 26219 log.cc:824] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:09.827040 26219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: No bootstrap required, opened a new log
I20260807 04:10:09.834424 26219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } }
I20260807 04:10:09.834820 26219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:09.834942 26219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 355f9a504f564ffd82691adf27dab598, State: Initialized, Role: FOLLOWER
I20260807 04:10:09.835528 26219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } }
I20260807 04:10:09.836432 26226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } } }
I20260807 04:10:09.836944 26226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:10:09.837669 26219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 04:10:09.847903 26237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:10:09.848083 26237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:10:09.850818 26148 master_runner.cc:428] Detected that this master 355f9a504f564ffd82691adf27dab598 is joining an existing cluster
I20260807 04:10:09.850950 26148 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.193.61:46661
I20260807 04:10:09.883080 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.864798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39876 (local address 127.18.193.61:46661)
0807 04:10:09.864985 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:09.864989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:09.866168 (+ 1179us) server_negotiation.cc:407] Connection header received
0807 04:10:09.866469 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:09.866477 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:09.866560 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:09.866654 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:09.868596 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.869799 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.871808 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.872739 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.872827 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.875675 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:09.875683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:09.875688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:09.875725 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:09.879688 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.880600 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.880606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.880610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.880794 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.881231 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.881236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.881241 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.881646 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:09.881792 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:09.882341 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 04:10:09.882619 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":62}
I20260807 04:10:09.883819 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.864602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51244 (local address 127.18.193.62:35583)
0807 04:10:09.864835 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:09.864843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:09.867305 (+ 2462us) server_negotiation.cc:407] Connection header received
0807 04:10:09.867496 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:09.867501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:09.867805 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:09.867926 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:09.869447 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.870551 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.872742 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.873409 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.873531 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.876765 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:09.876771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:09.876780 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:09.876826 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:09.880676 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.881480 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.881485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.881488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.881652 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.882092 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.882098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.882102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.882463 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:09.882619 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:09.883220 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 04:10:09.883463 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 04:10:09.884167 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.864861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39876)
0807 04:10:09.866015 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:09.866053 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:09.866285 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:09.866925 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:09.866929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:09.867917 (+ 988us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:09.868375 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.868389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.870019 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.870024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.871657 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.871672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.873134 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.873140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.873312 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.874072 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:09.874100 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:09.875469 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:09.879840 (+ 4371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.879846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.879849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.880428 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.880932 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.880937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.880940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.881106 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.881872 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:09.881878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:09.882049 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 04:10:09.883376 (+ 1327us) client_negotiation.cc:240] Negotiation successful
0807 04:10:09.883686 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":548,"threads_started":1}
I20260807 04:10:09.884853 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.865396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51244)
0807 04:10:09.867155 (+ 1759us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:09.867174 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:09.867340 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:09.868156 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:09.868171 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:09.868941 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:09.869260 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.869273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.870773 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.870778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.872520 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.872535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.874225 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.874231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.874433 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.875074 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:09.875097 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:09.876281 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:09.880850 (+ 4569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.880854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.880858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.881335 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.881792 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.881797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.881801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.881960 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.882703 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:09.882708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:09.882906 (+ 198us) client_negotiation.cc:769] Sending connection context
0807 04:10:09.884072 (+ 1166us) client_negotiation.cc:240] Negotiation successful
0807 04:10:09.884411 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1607,"mutex_wait_us":3,"thread_start_us":1400,"threads_started":1}
WARNING: no policy specified for oryx/127.18.193.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.60@KRBTEST.COM" created.
Aug 07 04:10:09 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075807, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.62@KRBTEST.COM for oryx/127.18.193.61@KRBTEST.COM
I20260807 04:10:09.909754 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.891826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.18.193.61:46661)
0807 04:10:09.892017 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:09.892026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:09.893325 (+ 1299us) server_negotiation.cc:407] Connection header received
0807 04:10:09.893732 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:09.893738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:09.893860 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:09.893999 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:09.895524 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.896841 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.898963 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.899724 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.899867 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.903652 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:09.903657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:09.903661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:09.903691 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:09.906699 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.907508 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.907516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.907520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.907721 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:09.908132 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:09.908138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:09.908141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:09.908452 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:09.908645 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:09.909107 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 04:10:09.909353 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:10:09.909813 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:09.891809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39882)
0807 04:10:09.893189 (+ 1380us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:09.893233 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:09.893470 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:09.894069 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:09.894079 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:09.895019 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:09.895343 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.895356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.896981 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.896989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.898782 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:09.898802 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:09.899930 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:09.899939 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:09.900201 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:09.901085 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:09.901108 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:09.903444 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:09.906880 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.906890 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.906897 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.907317 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.907837 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:09.907842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:09.907846 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:09.908008 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:09.908619 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:09.908628 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:09.908817 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 04:10:09.909008 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 04:10:09.909363 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1113,"thread_start_us":910,"threads_started":1}
I20260807 04:10:09.912240 25660 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.193.61:46661: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "355f9a504f564ffd82691adf27dab598"
member_type: NON_VOTER
last_known_addr {
host: "127.18.193.61"
port: 46661
}
attrs {
promote: true
}
}
dest_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8"
cas_config_opid_index: -1
I20260807 04:10:09.913275 25660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } }
I20260807 04:10:09.921581 26251 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 355f9a504f564ffd82691adf27dab598 (127.18.193.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:09.923206 26193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:09.929448 26193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Refusing update from remote peer 0d3b1030fdf24283adc4bb0f87f8dfc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 04:10:09.931017 26252 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.60.keytab.
Entry for principal oryx/127.18.193.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.60.keytab.
I20260807 04:10:09.926888 26251 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.193.61:46661
I20260807 04:10:09.933825 26148 master.cc:537] Master@127.18.193.61:46661 shutting down...
I20260807 04:10:09.934146 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:44361
--webserver_interface=127.18.193.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.60.keytab
--principal=oryx/127.18.193.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:10:09.934921 26250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:09.935359 26250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:09.961447 26193 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 355f9a504f564ffd82691adf27dab598 (127.18.193.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:09.980583 26261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:09.981078 26261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:10:09.981308 26226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d3b1030fdf24283adc4bb0f87f8dfc8. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:09.981572 26226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:10:09.981086 26257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:09.982203 26257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:10:09.984083 26148 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 LEARNER]: Raft consensus shutting down.
I20260807 04:10:09.984627 26148 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 LEARNER]: Raft consensus is shut down!
I20260807 04:10:09.984802 26148 tablet_replica.cc:330] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: stopping tablet replica
I20260807 04:10:10.004166 26148 master.cc:552] Master@127.18.193.61:46661 shutdown complete.
I20260807 04:10:10.004364 26148 master_runner.cc:305] Clearing existing system tablet
I20260807 04:10:10.004947 26148 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:10.005059 26148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:10.008351 26148 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 04:10:10.012279 26263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:10.012758 26148 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:10.013250 26148 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 04:10:10.013394 26148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
uuid: "355f9a504f564ffd82691adf27dab598"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "1c1465b5973525a5747c264b2f55770d"
server_key_iv: "f9cbe8e4e8645b1eb93786b9e481f669"
server_key_version: "encryptionkey@0"
I20260807 04:10:10.018751 26148 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:10.026439 26148 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 04:10:10.026678 26148 log.cc:1196] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 04:10:10.027233 26148 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Deleting consensus metadata
I20260807 04:10:10.027560 26148 master_runner.cc:315] Copying system tablet from 127.18.193.62:35583
I20260807 04:10:10.030834 26148 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: tablet copy: Beginning tablet copy session from remote peer at address 127.18.193.62:35583
I20260807 04:10:10.049084 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.031968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.18.193.62:35583)
0807 04:10:10.032188 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.032194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.032797 (+ 603us) server_negotiation.cc:407] Connection header received
0807 04:10:10.033014 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.033020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.033130 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.033260 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.034332 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.035496 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.037644 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.038546 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.038742 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.041518 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.041525 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.041530 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.041575 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.045587 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.046535 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.046543 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.046547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.046727 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.047252 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.047257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.047261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.047661 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.047838 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.048370 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.048659 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 04:10:10.049106 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.031682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51246)
0807 04:10:10.032657 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.032683 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.032852 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.033315 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.033320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.033842 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.034151 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.034165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.035679 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.035688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.037470 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.037491 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.038774 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.038780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.039023 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.039885 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.039913 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.041258 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.045774 (+ 4516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.045782 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.045786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.046367 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.046872 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.046877 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.046880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.047097 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.047816 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.047821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.048032 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.048209 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.048542 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":647,"threads_started":1}
I20260807 04:10:10.050485 25682 tablet_copy_service.cc:140] P 0d3b1030fdf24283adc4bb0f87f8dfc8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 355f9a504f564ffd82691adf27dab598 ({username='oryx', principal='oryx/127.18.193.61@KRBTEST.COM'} at 127.0.0.1:51246)
I20260807 04:10:10.050799 25682 tablet_copy_service.cc:161] P 0d3b1030fdf24283adc4bb0f87f8dfc8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 355f9a504f564ffd82691adf27dab598 at {username='oryx', principal='oryx/127.18.193.61@KRBTEST.COM'} at 127.0.0.1:51246: session id = 355f9a504f564ffd82691adf27dab598-00000000000000000000000000000000
I20260807 04:10:10.055547 25682 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 04:10:10.058069 26148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 04:10:10.065459 26148 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: tablet copy: Starting download of 0 data blocks...
I20260807 04:10:10.065826 26148 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: tablet copy: Starting download of 1 WAL segments...
I20260807 04:10:10.069159 26148 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 04:10:10.074446 25682 tablet_copy_service.cc:342] P 0d3b1030fdf24283adc4bb0f87f8dfc8: Request end of tablet copy session 355f9a504f564ffd82691adf27dab598-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.193.61@KRBTEST.COM'} at 127.0.0.1:51246
I20260807 04:10:10.074703 25682 tablet_copy_service.cc:433] P 0d3b1030fdf24283adc4bb0f87f8dfc8: ending tablet copy session 355f9a504f564ffd82691adf27dab598-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 355f9a504f564ffd82691adf27dab598
I20260807 04:10:10.087954 26148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:10.094911 26270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:10.097539 26271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:10.101824 26273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:10.106135 26148 server_base.cc:1034] running on GCE node
I20260807 04:10:10.106537 26148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:10.107299 26148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:10.108541 26148 hybrid_clock.cc:648] HybridClock initialized: now 1786075810108450 us; error 55 us; skew 500 ppm
I20260807 04:10:10.108800 26148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075810, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:10.111123 26148 init.cc:377] Logged in from keytab as oryx/127.18.193.61@KRBTEST.COM (short username oryx)
I20260807 04:10:10.115561 26148 webserver.cc:460] Webserver started at http://127.18.193.61:46797/ using document root <none> and password file <none>
I20260807 04:10:10.116051 26148 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:10.116168 26148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:10.119714 26148 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:10:10.121805 26280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:10.122246 26148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 04:10:10.122390 26148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
uuid: "355f9a504f564ffd82691adf27dab598"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "1c1465b5973525a5747c264b2f55770d"
server_key_iv: "f9cbe8e4e8645b1eb93786b9e481f669"
server_key_version: "encryptionkey@0"
I20260807 04:10:10.122565 26148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:10.137519 26148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:10.142421 26148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:10.185964 26148 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.61:46661
I20260807 04:10:10.185999 26336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.61:46661 every 8 connection(s)
I20260807 04:10:10.187624 26148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
I20260807 04:10:10.193686 26337 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:10:10.198746 26337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Bootstrap starting.
I20260807 04:10:10.221436 26337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:10:10.222078 26337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: Bootstrap complete.
I20260807 04:10:10.223255 26337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } }
I20260807 04:10:10.223577 26337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 355f9a504f564ffd82691adf27dab598, State: Initialized, Role: LEARNER
I20260807 04:10:10.223872 26337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } }
I20260807 04:10:10.224892 26340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:10.225232 26337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:10.225368 26340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:10:10.234040 26141 heartbeater.cc:503] Master 127.18.193.62:35583 was elected leader, sending a full tablet report...
I20260807 04:10:10.234360 26351 catalog_manager.cc:1259] Loaded cluster ID: e74ee0415bb5441bbdd3cc3eccdb9151
I20260807 04:10:10.234452 26351 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: loading cluster ID for follower catalog manager: success
I20260807 04:10:10.237640 26351 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: acquiring CA information for follower catalog manager: success
I20260807 04:10:10.240027 26351 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:10:10.478145 26250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:10.479274 26250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:10.496616 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.472242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39896)
0807 04:10:10.487901 (+ 15659us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.487927 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.488137 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.488915 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.488921 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.488954 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:10.489184 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.489196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.491668 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.491672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.493849 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.493865 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.495284 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.495291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.495701 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.495804 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.495913 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.496087 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15460,"thread_start_us":15995,"threads_started":1}
I20260807 04:10:10.497085 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.472720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.18.193.61:46661)
0807 04:10:10.477274 (+ 4554us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.477286 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.488047 (+ 10761us) server_negotiation.cc:407] Connection header received
0807 04:10:10.488294 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.488300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.488404 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.488573 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:10.489386 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.491371 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.494028 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.495053 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.495239 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.495589 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.495971 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.496121 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4227,"thread_start_us":3910,"threads_started":1}
I20260807 04:10:10.498498 26340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d3b1030fdf24283adc4bb0f87f8dfc8. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:10.498853 26340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:10:10.499845 26353 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: attempting to promote NON_VOTER 355f9a504f564ffd82691adf27dab598 to VOTER
I20260807 04:10:10.500964 26353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } }
I20260807 04:10:10.503454 26309 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 LEARNER]: Refusing update from remote peer 0d3b1030fdf24283adc4bb0f87f8dfc8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 04:10:10.504189 26353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 04:10:10.507849 26352 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 355f9a504f564ffd82691adf27dab598 (127.18.193.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } }
I20260807 04:10:10.508800 26309 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 355f9a504f564ffd82691adf27dab598 (127.18.193.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } }
I20260807 04:10:10.513655 26352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } }
I20260807 04:10:10.514119 26353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } }
I20260807 04:10:10.514423 26353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:10.514781 26340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } }
I20260807 04:10:10.515033 26340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:10:10.515492 26352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
W20260807 04:10:10.536194 26254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:10.536603 26254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:10.536787 26254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:10.546344 26254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:10.546530 26254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:10.546626 26254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:10.546748 26254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:10.546833 26254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:10.558812 26254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.60.keytab
--principal=oryx/127.18.193.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:44361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.193.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:10.560355 26254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:10.561745 26254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:10.576251 26362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:10.576324 26365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:10.577188 26363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:10.578650 26254 server_base.cc:1034] running on GCE node
I20260807 04:10:10.579581 26254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:10.581408 26254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:10.582777 26254 hybrid_clock.cc:648] HybridClock initialized: now 1786075810582627 us; error 59 us; skew 500 ppm
I20260807 04:10:10.583266 26254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075810, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:10.587848 26254 init.cc:377] Logged in from keytab as oryx/127.18.193.60@KRBTEST.COM (short username oryx)
I20260807 04:10:10.590148 26254 webserver.cc:460] Webserver started at http://127.18.193.60:41449/ using document root <none> and password file <none>
I20260807 04:10:10.590890 26254 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:10.591044 26254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:10.591373 26254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:10.595052 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/instance:
uuid: "b2ebdf88a8634958bc9c1a32301d99ee"
format_stamp: "Formatted at 2026-08-07 04:10:10 on dist-test-slave-8n49"
server_key: "bd8deac56c6dd7164bf2d259debb5453"
server_key_iv: "bce0fb07a1aa7d2063d6e129f3b8960e"
server_key_version: "encryptionkey@0"
I20260807 04:10:10.596015 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal/instance:
uuid: "b2ebdf88a8634958bc9c1a32301d99ee"
format_stamp: "Formatted at 2026-08-07 04:10:10 on dist-test-slave-8n49"
server_key: "bd8deac56c6dd7164bf2d259debb5453"
server_key_iv: "bce0fb07a1aa7d2063d6e129f3b8960e"
server_key_version: "encryptionkey@0"
I20260807 04:10:10.602681 26254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 04:10:10.607939 26373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:10.609767 26254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 04:10:10.610042 26254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
uuid: "b2ebdf88a8634958bc9c1a32301d99ee"
format_stamp: "Formatted at 2026-08-07 04:10:10 on dist-test-slave-8n49"
server_key: "bd8deac56c6dd7164bf2d259debb5453"
server_key_iv: "bce0fb07a1aa7d2063d6e129f3b8960e"
server_key_version: "encryptionkey@0"
I20260807 04:10:10.610350 26254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:10.637246 26254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:10.643908 26254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:10.644321 26254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:10.685586 26254 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.60:44361
I20260807 04:10:10.686064 26428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.60:44361 every 8 connection(s)
I20260807 04:10:10.687898 26254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
I20260807 04:10:10.693933 26429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:10.696224 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 26254
I20260807 04:10:10.709096 26429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.60@KRBTEST.COM
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075810, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.60@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:10.730631 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.699514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.18.193.60:44361)
0807 04:10:10.701218 (+ 1704us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.701242 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.701977 (+ 735us) server_negotiation.cc:407] Connection header received
0807 04:10:10.704719 (+ 2742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.704748 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.705245 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.705747 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.708615 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.710621 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.712731 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.714630 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.714886 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.721415 (+ 6529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.721426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.721437 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.721475 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.725658 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.726684 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.726692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.726698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.726858 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.727301 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.727305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.727307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.727978 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.728346 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.728923 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.729337 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1221,"thread_start_us":832,"threads_started":1}
I20260807 04:10:10.734450 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.713217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.18.193.62:35583)
0807 04:10:10.714127 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.714137 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.715379 (+ 1242us) server_negotiation.cc:407] Connection header received
0807 04:10:10.715608 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.715613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.715685 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.715909 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.717833 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.718701 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.721096 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.721636 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.721775 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.726695 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.726701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.726705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.726739 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.730580 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.731762 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.731766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.731768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.731985 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.732527 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.732531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.732533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.732941 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.733119 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.733611 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.733883 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":408,"threads_started":1}
I20260807 04:10:10.734505 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.713337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51262)
0807 04:10:10.715172 (+ 1835us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.715240 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.715482 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.716062 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.716071 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.717156 (+ 1085us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.717665 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.717675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.718880 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.718886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.720949 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.720962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.721947 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.721964 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.722237 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.723237 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.723254 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.726432 (+ 3178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.730725 (+ 4293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.730737 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.730764 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.731602 (+ 838us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.732162 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.732167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.732170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.732326 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.733084 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.733092 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.733294 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.733512 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.733856 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"thread_start_us":1051,"threads_started":1}
I20260807 04:10:10.736157 26429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075810, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.60@KRBTEST.COM for oryx/127.18.193.61@KRBTEST.COM
I20260807 04:10:10.751664 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.736889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39900)
0807 04:10:10.737075 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.737091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.737204 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.737600 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.737604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.738062 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.738250 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.738259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.739351 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.739356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.740408 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.740440 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.741375 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.741381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.741572 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.742212 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.742228 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.745023 (+ 2795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.748474 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.748479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.748483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.749069 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.749690 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.749696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.749700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.749889 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.750502 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.750507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.750720 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.750873 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.751252 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 04:10:10.751782 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.737104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.18.193.61:46661)
0807 04:10:10.737316 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.737322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.737338 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:10:10.737393 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.737397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.737455 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.737582 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.738396 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.739206 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.740617 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.741207 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.741344 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.745239 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.745244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.745247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.745281 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.748317 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.749355 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.749362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.749366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.749559 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.750009 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.750014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.750017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.750378 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.750547 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.751018 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.751313 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:10:10.753296 26429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075810, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.60@KRBTEST.COM for oryx/127.18.193.60@KRBTEST.COM
I20260807 04:10:10.769032 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.754450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.18.193.60:44361)
0807 04:10:10.754633 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.754638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.754657 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:10:10.754737 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.754744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.754801 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.754937 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.755697 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.756375 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.757675 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.758186 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.758350 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.762155 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.762161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.762164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.762193 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.765791 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.766603 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.766611 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.766614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.766764 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.767273 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.767280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.767283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.767560 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.767714 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.768252 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.768648 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":79}
I20260807 04:10:10.769134 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.754110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.0.0.1:44210)
0807 04:10:10.754455 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.754473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.754627 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.754952 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.754956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.755401 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.755550 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.755558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.756506 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.756514 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.757532 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.757545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.758406 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.758412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.758754 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.759491 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.759510 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.761928 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.765966 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.765974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.765978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.766482 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.766922 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.766929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.766933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.767157 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.767675 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.767679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.767968 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.768143 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.768469 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 04:10:10.775252 26429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Bootstrap starting.
I20260807 04:10:10.779691 26429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:10.781213 26429 log.cc:824] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:10.784415 26429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: No bootstrap required, opened a new log
I20260807 04:10:10.794310 26429 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } }
I20260807 04:10:10.794700 26429 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:10.794926 26429 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2ebdf88a8634958bc9c1a32301d99ee, State: Initialized, Role: FOLLOWER
I20260807 04:10:10.795579 26429 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } }
I20260807 04:10:10.797258 26436 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } } }
I20260807 04:10:10.797856 26436 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:10:10.798332 26429 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: configured and running, proceeding with master startup.
W20260807 04:10:10.812947 26447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:10:10.813210 26447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:10:10.816581 26254 master_runner.cc:428] Detected that this master b2ebdf88a8634958bc9c1a32301d99ee is joining an existing cluster
I20260807 04:10:10.816771 26254 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.193.60:44361
I20260807 04:10:10.838884 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 25726
I20260807 04:10:10.850749 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.823879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39902)
0807 04:10:10.825293 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.825327 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.825669 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.826232 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.826237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.827453 (+ 1216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.828301 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.828320 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.831273 (+ 2953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.831282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.834781 (+ 3499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.834800 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.836648 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.836657 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.837023 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.838013 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.838039 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.839893 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.844814 (+ 4921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.844819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.844823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.845939 (+ 1116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.846550 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.846558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.846564 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.847585 (+ 1021us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.848605 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.848610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.849121 (+ 511us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.849467 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.849954 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1116,"thread_start_us":893,"threads_started":1}
I20260807 04:10:10.852542 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.824444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.18.193.61:46661)
0807 04:10:10.824731 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.824740 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.825594 (+ 854us) server_negotiation.cc:407] Connection header received
0807 04:10:10.825849 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.825858 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.826029 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.826179 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.828536 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.830336 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.835075 (+ 4739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.836100 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.836246 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.840146 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.840153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.840159 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.840206 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.844620 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.846127 (+ 1507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.846137 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.846144 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.846376 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.847758 (+ 1382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.847763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.847766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.848324 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.848482 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.851466 (+ 2984us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.851938 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260807 04:10:10.857223 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.823444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.18.193.62:35583)
0807 04:10:10.823771 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.823777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.829869 (+ 6092us) server_negotiation.cc:407] Connection header received
0807 04:10:10.830284 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.830290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.830408 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.830492 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.834950 (+ 4458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.836644 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.840237 (+ 3593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.841282 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.841466 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.846806 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.846816 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.846824 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.846880 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.852274 (+ 5394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.853452 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.853458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.853461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.853730 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.854329 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.854334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.854337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.854928 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.855077 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.856212 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.856600 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260807 04:10:10.859264 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.825264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51278)
0807 04:10:10.829617 (+ 4353us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.829649 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.830082 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.830817 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.830823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.833384 (+ 2561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.834644 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.834668 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.836873 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.836880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.840045 (+ 3165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.840062 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.841943 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.841953 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.842310 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.843282 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.843314 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.845528 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.852509 (+ 6981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.852518 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.852525 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.853282 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.853910 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.853918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.853921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.854189 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.855173 (+ 984us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.855179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.855798 (+ 619us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.857433 (+ 1635us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.858035 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4113,"mutex_wait_us":24,"thread_start_us":3803,"threads_started":1}
I20260807 04:10:10.876355 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.830756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.18.193.60:44361)
0807 04:10:10.831089 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.831099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.845475 (+ 14376us) server_negotiation.cc:407] Connection header received
0807 04:10:10.845745 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.845750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.845841 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.845925 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.853757 (+ 7832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.855459 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.858363 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.859602 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.859751 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.866177 (+ 6426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.866186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.866191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.866230 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.871323 (+ 5093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.872871 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.872877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.872880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.873078 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.873591 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.873596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.873599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.874161 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.874403 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.875462 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.875792 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.1@KRBTEST.COM
I20260807 04:10:10.886109 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.829744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.0.0.1:44214)
0807 04:10:10.842681 (+ 12937us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.845278 (+ 2597us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.845585 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.846112 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.846116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.847254 (+ 1138us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.853570 (+ 6316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.853599 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.855787 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.855795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.858180 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.858197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.859856 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.859865 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.860166 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.864013 (+ 3847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.864039 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.865929 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.871702 (+ 5773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.871716 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.871728 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.872741 (+ 1013us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.873255 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.873260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.873264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.873476 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.874486 (+ 1010us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.874490 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.874742 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.884660 (+ 9918us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.885199 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11461,"thread_start_us":7646,"threads_started":1}
Aug 07 04:10:10 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075807, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.62@KRBTEST.COM for oryx/127.18.193.60@KRBTEST.COM
I20260807 04:10:10.892287 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.840969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.18.193.1:45067)
0807 04:10:10.860152 (+ 19183us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.860162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.860195 (+ 33us) server_negotiation.cc:407] Connection header received
0807 04:10:10.860274 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.860284 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.860672 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.861004 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.863811 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.866045 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.869273 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.870803 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.880334 (+ 9531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.881007 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.881027 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.881053 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.881127 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.886991 (+ 5864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.888292 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.888307 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.888310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.888528 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.889079 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.889086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.889094 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.889514 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.890052 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.890552 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.890948 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18970,"thread_start_us":18713,"threads_started":1}
I20260807 04:10:10.893977 25853 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 04:10:10.898038 26379 tablet.cc:2369] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:10:10.904027 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.867042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.18.193.60:44361)
0807 04:10:10.877113 (+ 10071us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:10.877119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:10.877141 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:10:10.877249 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:10.877254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:10.877392 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:10.879224 (+ 1832us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:10.879619 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.881159 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.884412 (+ 3253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.885977 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.891286 (+ 5309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.893017 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:10.893025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:10.893030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:10.893152 (+ 122us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:10.899964 (+ 6812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.901439 (+ 1475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.901443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.901445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.901604 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:10.902115 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:10.902120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:10.902124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:10.902518 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:10.902681 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:10.903247 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 04:10:10.903514 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9959,"thread_start_us":15608,"threads_started":1}
I20260807 04:10:10.904085 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:10.865873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.0.0.1:44222)
0807 04:10:10.866279 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:10.866314 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:10.866447 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:10.877690 (+ 11243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:10.877697 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:10.878585 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:10.878939 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.878957 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.881412 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.881419 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.884142 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:10.884163 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:10.887095 (+ 2932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:10.887109 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:10.887376 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:10.888997 (+ 1621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:10.889023 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:10.892788 (+ 3765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:10.900252 (+ 7464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.900262 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.900269 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.901266 (+ 997us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.901740 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:10.901745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:10.901749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:10.901977 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:10.902661 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:10.902666 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:10.902944 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 04:10:10.903121 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 04:10:10.903495 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289}
I20260807 04:10:10.906031 26436 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:10.906623 26436 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } }
I20260807 04:10:10.907269 25660 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.193.60:44361: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee"
member_type: NON_VOTER
last_known_addr {
host: "127.18.193.60"
port: 44361
}
attrs {
promote: true
}
}
dest_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8"
cas_config_opid_index: 6
I20260807 04:10:10.908124 25660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } }
I20260807 04:10:10.909632 26436 leader_election.cc:288] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583), 355f9a504f564ffd82691adf27dab598 (127.18.193.61:46661)
I20260807 04:10:10.910015 26309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "355f9a504f564ffd82691adf27dab598" is_pre_election: true
I20260807 04:10:10.911253 26308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Refusing update from remote peer 0d3b1030fdf24283adc4bb0f87f8dfc8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 04:10:10.911988 25671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" is_pre_election: true
I20260807 04:10:10.913147 26352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 04:10:10.913192 26377 leader_election.cc:302] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2ebdf88a8634958bc9c1a32301d99ee; no voters: 0d3b1030fdf24283adc4bb0f87f8dfc8, 355f9a504f564ffd82691adf27dab598
I20260807 04:10:10.913796 26436 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 04:10:10.914994 26402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:10.919747 26352 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER b2ebdf88a8634958bc9c1a32301d99ee (127.18.193.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:10.920378 26308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER b2ebdf88a8634958bc9c1a32301d99ee (127.18.193.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:10.925469 26340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:10.925778 26340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:10:10.926815 26402 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 FOLLOWER]: Refusing update from remote peer 0d3b1030fdf24283adc4bb0f87f8dfc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 04:10:10.927624 26353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:10.927529 26352 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.193.60:44361
I20260807 04:10:10.930444 26254 master.cc:537] Master@127.18.193.60:44361 shutting down...
I20260807 04:10:10.934859 26352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:10.934890 26250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:10.937472 26250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:10.937520 26352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:10.941486 26402 raft_consensus.cc:858] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } }
W20260807 04:10:10.941797 26402 raft_consensus.cc:1565] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } }
I20260807 04:10:10.950449 26402 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 355f9a504f564ffd82691adf27dab598 (127.18.193.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:10.961056 26465 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:10.961719 26465 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 04:10:10.962023 26461 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:10.962404 26461 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 04:10:10.963972 26352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:10.964282 26352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:10.964581 26436 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d3b1030fdf24283adc4bb0f87f8dfc8. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: NON_VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: true } } }
I20260807 04:10:10.964830 26436 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 04:10:10.965544 26254 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
W20260807 04:10:10.965960 25645 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 04:10:10.966045 26254 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 04:10:10.966230 26254 tablet_replica.cc:330] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: stopping tablet replica
I20260807 04:10:10.975765 26254 master.cc:552] Master@127.18.193.60:44361 shutdown complete.
I20260807 04:10:10.975874 26254 master_runner.cc:305] Clearing existing system tablet
I20260807 04:10:10.976322 26254 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:10.976430 26254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260807 04:10:10.984182 25645 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 -> Peer b2ebdf88a8634958bc9c1a32301d99ee (127.18.193.60:44361): Couldn't send request to peer b2ebdf88a8634958bc9c1a32301d99ee. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 04:10:10.985074 26254 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.002s
I20260807 04:10:10.986971 26468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:10.987525 26254 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:10.988123 26254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:10:10.988282 26254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
uuid: "b2ebdf88a8634958bc9c1a32301d99ee"
format_stamp: "Formatted at 2026-08-07 04:10:10 on dist-test-slave-8n49"
server_key: "bd8deac56c6dd7164bf2d259debb5453"
server_key_iv: "bce0fb07a1aa7d2063d6e129f3b8960e"
server_key_version: "encryptionkey@0"
I20260807 04:10:10.993180 26254 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:11.000406 26254 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 04:10:11.000700 26254 log.cc:1196] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 04:10:11.001382 26254 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Deleting consensus metadata
I20260807 04:10:11.001744 26254 master_runner.cc:315] Copying system tablet from 127.18.193.62:35583
I20260807 04:10:11.004410 26254 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: tablet copy: Beginning tablet copy session from remote peer at address 127.18.193.62:35583
I20260807 04:10:11.028012 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.005331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51284)
0807 04:10:11.013007 (+ 7676us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:11.013031 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:11.013219 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:11.013657 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:11.013662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:11.014251 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:11.014549 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.014565 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.016069 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.016080 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.017706 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.017724 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.018672 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.018679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.018881 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.019647 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:11.019674 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:11.020857 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:11.024939 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.024949 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.024954 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.025504 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.025971 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.025977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.025980 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.026165 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.026842 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:11.026847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:11.027041 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 04:10:11.027188 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 04:10:11.027536 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7530,"thread_start_us":7383,"threads_started":1}
I20260807 04:10:11.028116 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.005428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.18.193.62:35583)
0807 04:10:11.005681 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:11.005688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:11.013172 (+ 7484us) server_negotiation.cc:407] Connection header received
0807 04:10:11.013384 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:11.013390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:11.013481 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:11.013596 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:11.014727 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.015901 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.017864 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.018501 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.018627 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.021086 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:11.021102 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:11.021106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:11.021147 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:11.024797 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.025652 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.025659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.025662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.025821 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.026349 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.026358 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.026361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.026709 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:11.026855 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:11.027343 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 04:10:11.027565 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 04:10:11.029268 25682 tablet_copy_service.cc:140] P 0d3b1030fdf24283adc4bb0f87f8dfc8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b2ebdf88a8634958bc9c1a32301d99ee ({username='oryx', principal='oryx/127.18.193.60@KRBTEST.COM'} at 127.0.0.1:51284)
I20260807 04:10:11.029450 25682 tablet_copy_service.cc:161] P 0d3b1030fdf24283adc4bb0f87f8dfc8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b2ebdf88a8634958bc9c1a32301d99ee at {username='oryx', principal='oryx/127.18.193.60@KRBTEST.COM'} at 127.0.0.1:51284: session id = b2ebdf88a8634958bc9c1a32301d99ee-00000000000000000000000000000000
I20260807 04:10:11.033252 25682 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 04:10:11.035226 26254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 04:10:11.035356 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 25726
I20260807 04:10:11.042343 26254 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: tablet copy: Starting download of 0 data blocks...
I20260807 04:10:11.042603 26254 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: tablet copy: Starting download of 1 WAL segments...
I20260807 04:10:11.044924 26254 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 04:10:11.048131 25682 tablet_copy_service.cc:342] P 0d3b1030fdf24283adc4bb0f87f8dfc8: Request end of tablet copy session b2ebdf88a8634958bc9c1a32301d99ee-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.193.60@KRBTEST.COM'} at 127.0.0.1:51284
I20260807 04:10:11.048290 25682 tablet_copy_service.cc:433] P 0d3b1030fdf24283adc4bb0f87f8dfc8: ending tablet copy session b2ebdf88a8634958bc9c1a32301d99ee-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b2ebdf88a8634958bc9c1a32301d99ee
I20260807 04:10:11.062978 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:45067
--local_ip_for_outbound_sockets=127.18.193.1
--tserver_master_addrs=127.18.193.62:35583
--webserver_port=37989
--webserver_interface=127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab
--principal=oryx/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:10:11.064244 26254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:11.067476 26477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:11.078781 26481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:11.079159 26478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:11.079450 26254 server_base.cc:1034] running on GCE node
I20260807 04:10:11.079777 26254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:11.080468 26254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:11.081804 26254 hybrid_clock.cc:648] HybridClock initialized: now 1786075811081757 us; error 41 us; skew 500 ppm
I20260807 04:10:11.082057 26254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:11 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075811, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:11.084182 26254 init.cc:377] Logged in from keytab as oryx/127.18.193.60@KRBTEST.COM (short username oryx)
I20260807 04:10:11.089661 26254 webserver.cc:460] Webserver started at http://127.18.193.60:46039/ using document root <none> and password file <none>
I20260807 04:10:11.090054 26254 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:11.090195 26254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:11.093220 26254 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 04:10:11.097924 26488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:11.098439 26254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 04:10:11.098624 26254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
uuid: "b2ebdf88a8634958bc9c1a32301d99ee"
format_stamp: "Formatted at 2026-08-07 04:10:10 on dist-test-slave-8n49"
server_key: "bd8deac56c6dd7164bf2d259debb5453"
server_key_iv: "bce0fb07a1aa7d2063d6e129f3b8960e"
server_key_version: "encryptionkey@0"
I20260807 04:10:11.098855 26254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:11.111002 26254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:11.115211 26254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:11.161662 26254 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.60:44361
I20260807 04:10:11.161681 26541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.60:44361 every 8 connection(s)
I20260807 04:10:11.162678 26254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
I20260807 04:10:11.170689 26542 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:10:11.178750 26542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Bootstrap starting.
I20260807 04:10:11.209120 26542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:10:11.209825 26542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Bootstrap complete.
I20260807 04:10:11.210763 26542 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } }
I20260807 04:10:11.211021 26542 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b2ebdf88a8634958bc9c1a32301d99ee, State: Initialized, Role: LEARNER
I20260807 04:10:11.211272 26542 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } }
I20260807 04:10:11.212168 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:11.212495 26542 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:11.212649 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: LEARNER
I20260807 04:10:11.221611 26557 catalog_manager.cc:1259] Loaded cluster ID: e74ee0415bb5441bbdd3cc3eccdb9151
I20260807 04:10:11.221741 26557 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: loading cluster ID for follower catalog manager: success
I20260807 04:10:11.224370 26557 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: acquiring CA information for follower catalog manager: success
I20260807 04:10:11.226366 26557 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:10:11.460287 26250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:11.460723 26250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:11.468981 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.460524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.0.0.1:44234)
0807 04:10:11.461636 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:11.461654 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:11.461798 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:11.462251 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:11.462254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:11.462277 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:11.462483 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.462490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.464817 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.464823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.466179 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.466195 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.468156 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.468161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.468309 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.468344 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:10:11.468423 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 04:10:11.468537 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"thread_start_us":848,"threads_started":1}
I20260807 04:10:11.469141 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.460780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44234 (local address 127.18.193.60:44361)
0807 04:10:11.461487 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:11.461495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:11.461765 (+ 270us) server_negotiation.cc:407] Connection header received
0807 04:10:11.461962 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:11.461967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:11.462071 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:11.462229 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:11.462669 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.464644 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.466380 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.467914 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.468095 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.468421 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:11.468491 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 04:10:11.468722 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":316,"threads_started":1}
I20260807 04:10:11.470116 26516 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260807 04:10:11.470674 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d3b1030fdf24283adc4bb0f87f8dfc8. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: NON_VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: true } } }
I20260807 04:10:11.470978 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: LEARNER
I20260807 04:10:11.471500 26353 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: attempting to promote NON_VOTER b2ebdf88a8634958bc9c1a32301d99ee to VOTER
I20260807 04:10:11.472442 26353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } }
I20260807 04:10:11.474800 26516 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 LEARNER]: Refusing update from remote peer 0d3b1030fdf24283adc4bb0f87f8dfc8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 04:10:11.474887 26308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Refusing update from remote peer 0d3b1030fdf24283adc4bb0f87f8dfc8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 04:10:11.475536 26352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 04:10:11.475888 26353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 04:10:11.478724 26250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b2ebdf88a8634958bc9c1a32301d99ee (127.18.193.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.479487 26516 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b2ebdf88a8634958bc9c1a32301d99ee (127.18.193.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.479542 26308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b2ebdf88a8634958bc9c1a32301d99ee (127.18.193.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.483516 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.484268 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:10:11.484865 26352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.485234 26352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:11.485988 26352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.486353 26352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:11.485657 26250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.486817 26250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:11.489013 26562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:11.489344 26562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 04:10:11.510154 26480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:11.510455 26480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:11.510551 26480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:11.518509 26480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:11.518627 26480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:11.518743 26480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:11.529431 26480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab
--principal=oryx/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:45067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=37989
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:11.530838 26480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:11.532152 26480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:11.546205 26567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:11.546161 26568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:11.546254 26570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:11.547238 26480 server_base.cc:1034] running on GCE node
I20260807 04:10:11.547889 26480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:11.548913 26480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:11.550149 26480 hybrid_clock.cc:648] HybridClock initialized: now 1786075811550107 us; error 52 us; skew 500 ppm
I20260807 04:10:11.550472 26480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:11 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075811, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:11.553862 26480 init.cc:377] Logged in from keytab as oryx/127.18.193.1@KRBTEST.COM (short username oryx)
I20260807 04:10:11.555652 26480 webserver.cc:460] Webserver started at http://127.18.193.1:37989/ using document root <none> and password file <none>
I20260807 04:10:11.556453 26480 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:11.556661 26480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:11.562922 26480 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 04:10:11.566054 26577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:11.567497 26480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:11.567691 26480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "64c3784d514fc88f62c56d7c1b408e23"
server_key_iv: "4d96b6379f983791511df55728da221a"
server_key_version: "encryptionkey@0"
I20260807 04:10:11.568853 26480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:11.590976 26480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:11.595851 26480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:11.596181 26480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:11.597360 26480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:11.598771 26480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:11.598928 26480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:11.599036 26480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:11.599128 26480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:11.638584 26480 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:45067
I20260807 04:10:11.638671 26692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:45067 every 8 connection(s)
I20260807 04:10:11.640157 26480 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:11.647320 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 26480
I20260807 04:10:11.647763 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 25865
Aug 07 04:10:11 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075811, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
Aug 07 04:10:11 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075811, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.60@KRBTEST.COM
Aug 07 04:10:11 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075811, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.61@KRBTEST.COM
I20260807 04:10:11.675441 26697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.643425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:32991 (local address 127.18.193.62:35583)
0807 04:10:11.643886 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:11.643891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:11.645283 (+ 1392us) server_negotiation.cc:407] Connection header received
0807 04:10:11.648237 (+ 2954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:11.648243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:11.648341 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:11.648434 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:11.651183 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.652850 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.656424 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.657379 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.657507 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.663071 (+ 5564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:11.663078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:11.663083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:11.663133 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:11.667217 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.671096 (+ 3879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.671105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.671112 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.671279 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.672077 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.672095 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.672098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.672652 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:11.672842 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:11.674637 (+ 1795us) server_negotiation.cc:299] Negotiation successful
0807 04:10:11.674968 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":222,"threads_started":1}
I20260807 04:10:11.678294 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.644010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:32991)
0807 04:10:11.644911 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:11.644990 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:11.648077 (+ 3087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:11.648917 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:11.648929 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:11.649877 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:11.650900 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.650937 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.653126 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.653135 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.656145 (+ 3010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.656170 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.657889 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.657898 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.658223 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.659141 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:11.659175 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:11.662766 (+ 3591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:11.668585 (+ 5819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.668594 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.668612 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.670874 (+ 2262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.671514 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.671522 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.671527 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.671889 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.672930 (+ 1041us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:11.672963 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:11.673615 (+ 652us) client_negotiation.cc:769] Sending connection context
0807 04:10:11.673978 (+ 363us) client_negotiation.cc:240] Negotiation successful
0807 04:10:11.676589 (+ 2611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"spinlock_wait_cycles":32128,"thread_start_us":796,"threads_started":1}
I20260807 04:10:11.679025 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.646215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:53253 (local address 127.18.193.60:44361)
0807 04:10:11.646431 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:11.646439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:11.657767 (+ 11328us) server_negotiation.cc:407] Connection header received
0807 04:10:11.658245 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:11.658251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:11.658349 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:11.658429 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:11.659942 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.661052 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.663033 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.663702 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.663835 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.670321 (+ 6486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:11.670328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:11.670331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:11.670369 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:11.675074 (+ 4705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.676217 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.676223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.676226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.676429 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.676965 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.676970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.676973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.677432 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:11.677575 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:11.678259 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 04:10:11.678603 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:10:11.680065 26702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.645997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.18.193.1:53253)
0807 04:10:11.657644 (+ 11647us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:11.657669 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:11.658067 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:11.658686 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:11.658691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:11.659333 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:11.659745 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.659759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.661242 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.661247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.662866 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.662880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.664188 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.664195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.664391 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.665053 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:11.665077 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:11.669968 (+ 4891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:11.675242 (+ 5274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.675247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.675250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.676051 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.676597 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.676601 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.676604 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.676827 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.677648 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:11.677653 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:11.677839 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 04:10:11.677974 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 04:10:11.679498 (+ 1524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11503,"thread_start_us":11803,"threads_started":1}
I20260807 04:10:11.680729 26694 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35583
I20260807 04:10:11.681228 26694 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:11.682085 26693 heartbeater.cc:347] Connected to a master server at 127.18.193.60:44361
I20260807 04:10:11.682338 26693 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:11.682828 26694 heartbeater.cc:511] Master 127.18.193.62:35583 requested a full tablet report, sending...
I20260807 04:10:11.683388 26693 heartbeater.cc:511] Master 127.18.193.60:44361 requested a full tablet report, sending...
I20260807 04:10:11.684870 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.644697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.18.193.1:36907)
0807 04:10:11.645361 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:11.645378 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:11.647704 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:11.660745 (+ 13041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:11.660751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:11.661374 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:11.661756 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.661770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.663800 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.663808 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.665935 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:11.665956 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.667559 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.667569 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:11.667749 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.668312 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:11.668334 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:11.672666 (+ 4332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:11.679506 (+ 6840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.679512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.679516 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.680054 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.681002 (+ 948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:11.681008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:11.681011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:11.681277 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:11.682933 (+ 1656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:11.682947 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:11.683150 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 04:10:11.683296 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 04:10:11.683725 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"spinlock_wait_cycles":7424,"thread_start_us":5518,"threads_started":1}
I20260807 04:10:11.685616 25660 ts_manager.cc:194] Re-registered known tserver with Master: 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067)
I20260807 04:10:11.687140 25660 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:32991
I20260807 04:10:11.687589 26505 ts_manager.cc:194] Registered new tserver with Master: 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067)
I20260807 04:10:11.688287 26701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.644744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:36907 (local address 127.18.193.61:46661)
0807 04:10:11.660148 (+ 15404us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:11.660158 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:11.660184 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:10:11.660258 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:11.660267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:11.660364 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:11.661953 (+ 1589us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:11.662222 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.663585 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.666175 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.666950 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.667177 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.673306 (+ 6129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:11.673315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:11.673320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:11.673386 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:11.679223 (+ 5837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.680203 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.680209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.680213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.680453 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.681453 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.681459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.681464 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.681923 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:11.682876 (+ 953us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:11.684425 (+ 1549us) server_negotiation.cc:299] Negotiation successful
0807 04:10:11.684695 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15186,"thread_start_us":15779,"threads_started":1}
I20260807 04:10:11.690021 26695 heartbeater.cc:347] Connected to a master server at 127.18.193.61:46661
I20260807 04:10:11.690331 26695 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:11.691638 26695 heartbeater.cc:511] Master 127.18.193.61:46661 requested a full tablet report, sending...
Aug 07 04:10:11 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.2@KRBTEST.COM
I20260807 04:10:11.694060 26297 ts_manager.cc:194] Registered new tserver with Master: 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067)
I20260807 04:10:11.703007 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:11.649032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.18.193.2:33879)
0807 04:10:11.668112 (+ 19080us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:11.668124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:11.668159 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:10:11.668237 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:11.668252 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:11.669770 (+ 1518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:11.670106 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:11.674812 (+ 4706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.677493 (+ 2681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.681715 (+ 4222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:11.682961 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:11.683100 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:11.694301 (+ 11201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:11.694322 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:11.694341 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:11.694411 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:11.698780 (+ 4369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.700036 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.700044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.700046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.700284 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:11.700851 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:11.700855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:11.700857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:11.701326 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:11.701661 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:11.702276 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 04:10:11.702552 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18822,"thread_start_us":18538,"threads_started":1}
I20260807 04:10:11.704628 25996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:11.792991 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 25865
I20260807 04:10:11.812945 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:33879
--local_ip_for_outbound_sockets=127.18.193.2
--tserver_master_addrs=127.18.193.62:35583
--webserver_port=36187
--webserver_interface=127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab
--principal=oryx/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:12.183782 26706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:12.184142 26706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:12.184314 26706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:12.192219 26706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:12.192400 26706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:12.192647 26706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:12.203188 26706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab
--principal=oryx/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:33879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=36187
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:12.204716 26706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:12.206174 26706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:12.221817 26714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:12.221763 26717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:12.221792 26715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:12.222733 26706 server_base.cc:1034] running on GCE node
I20260807 04:10:12.223490 26706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:12.224645 26706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:12.225940 26706 hybrid_clock.cc:648] HybridClock initialized: now 1786075812225879 us; error 61 us; skew 500 ppm
I20260807 04:10:12.226342 26706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:12 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:12.230286 26706 init.cc:377] Logged in from keytab as oryx/127.18.193.2@KRBTEST.COM (short username oryx)
I20260807 04:10:12.231977 26706 webserver.cc:460] Webserver started at http://127.18.193.2:36187/ using document root <none> and password file <none>
I20260807 04:10:12.232806 26706 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:12.232978 26706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:12.239102 26706 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 04:10:12.242736 26724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:12.244166 26706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:10:12.244380 26706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "474dd01821e746dfbdcf7fd31daaa13a"
format_stamp: "Formatted at 2026-08-07 04:10:08 on dist-test-slave-8n49"
server_key: "eb4a872f29e3b0d19dacedf8c4219d93"
server_key_iv: "181fef265f3b57096628e67d64deef78"
server_key_version: "encryptionkey@0"
I20260807 04:10:12.245487 26706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:12.275588 26706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:12.280692 26706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:12.281029 26706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:12.282176 26706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:12.283704 26706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:12.283840 26706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:12.284006 26706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:12.284109 26706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:12.325991 26706 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:33879
I20260807 04:10:12.326062 26841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:33879 every 8 connection(s)
I20260807 04:10:12.327628 26706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:12.329985 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 26706
I20260807 04:10:12.330302 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 26007
Aug 07 04:10:12 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.61@KRBTEST.COM
Aug 07 04:10:12 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
Aug 07 04:10:12 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.3@KRBTEST.COM
I20260807 04:10:12.380009 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:12.331604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:53963 (local address 127.18.193.62:35583)
0807 04:10:12.349675 (+ 18071us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:12.349684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:12.355810 (+ 6126us) server_negotiation.cc:407] Connection header received
0807 04:10:12.356170 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:12.356188 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:12.356323 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:12.356445 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:12.358430 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.359917 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.362527 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.363534 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.363661 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:12.370007 (+ 6346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:12.370015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:12.370022 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:12.370072 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:12.374741 (+ 4669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.376122 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.376129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.376132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.376356 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.377046 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.377051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.377054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.377528 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:12.377704 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:12.379242 (+ 1538us) server_negotiation.cc:299] Negotiation successful
0807 04:10:12.379489 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16914,"thread_start_us":16798,"threads_started":1}
I20260807 04:10:12.382558 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:12.331562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:48397 (local address 127.18.193.61:46661)
0807 04:10:12.353823 (+ 22261us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:12.353828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:12.353846 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:10:12.353913 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:12.353917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:12.353998 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:12.354282 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:12.357214 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.358511 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.361205 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.362052 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.362180 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:12.367965 (+ 5785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:12.367975 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:12.367984 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:12.368041 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:12.377484 (+ 9443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.378732 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.378740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.378745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.378949 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.379486 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.379492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.379498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.379942 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:12.380235 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:12.381619 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0807 04:10:12.381918 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22153,"thread_start_us":22055,"threads_started":1}
I20260807 04:10:12.384359 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:12.331966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.2:53963)
0807 04:10:12.355672 (+ 23706us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:12.355696 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:12.355949 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:12.356964 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:12.356972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:12.357784 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:12.358172 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:12.358188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.360160 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.360168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:12.362279 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:12.362301 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.364143 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.364152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:12.364515 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:12.365463 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:12.365507 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:12.369736 (+ 4229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:12.374987 (+ 5251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:12.374997 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:12.375018 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:12.375970 (+ 952us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:12.376583 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:12.376589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:12.376592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:12.376911 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:12.377802 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:12.377811 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:12.378679 (+ 868us) client_negotiation.cc:769] Sending connection context
0807 04:10:12.381137 (+ 2458us) client_negotiation.cc:240] Negotiation successful
0807 04:10:12.381992 (+ 855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23498,"mutex_wait_us":34,"spinlock_wait_cycles":25984,"thread_start_us":22948,"threads_started":1}
I20260807 04:10:12.384409 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:12.331966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.18.193.2:48397)
0807 04:10:12.344048 (+ 12082us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:12.344131 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:12.347688 (+ 3557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:12.354670 (+ 6982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:12.354684 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:12.355890 (+ 1206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:12.356941 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:12.356975 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.358716 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.358724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:12.360987 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:12.361002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.362501 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.362509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:12.362799 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:12.363657 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:12.363695 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:12.367532 (+ 3837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:12.377781 (+ 10249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:12.377788 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:12.377792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:12.378511 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:12.379126 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:12.379131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:12.379135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:12.379340 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:12.380325 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:12.380330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:12.380628 (+ 298us) client_negotiation.cc:769] Sending connection context
0807 04:10:12.380822 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 04:10:12.383158 (+ 2336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11603,"mutex_wait_us":1487,"thread_start_us":16453,"threads_started":1}
Aug 07 04:10:12 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.60@KRBTEST.COM
I20260807 04:10:12.386974 26843 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35583
I20260807 04:10:12.387414 26843 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:12.388804 26843 heartbeater.cc:511] Master 127.18.193.62:35583 requested a full tablet report, sending...
I20260807 04:10:12.389842 26844 heartbeater.cc:347] Connected to a master server at 127.18.193.61:46661
I20260807 04:10:12.389997 26844 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:12.390797 26844 heartbeater.cc:511] Master 127.18.193.61:46661 requested a full tablet report, sending...
I20260807 04:10:12.390904 25660 ts_manager.cc:194] Re-registered known tserver with Master: 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879)
I20260807 04:10:12.392457 26297 ts_manager.cc:194] Registered new tserver with Master: 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879)
I20260807 04:10:12.392478 25660 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:53963
I20260807 04:10:12.397198 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:12.361938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.18.193.2:37311)
0807 04:10:12.371541 (+ 9603us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:12.371564 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:12.371802 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:12.374513 (+ 2711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:12.374518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:12.375124 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:12.375949 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:12.375965 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.377807 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.377812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:12.380736 (+ 2924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:12.380753 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.382490 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.382499 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:12.382876 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:12.383692 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:12.383714 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:12.387966 (+ 4252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:12.393419 (+ 5453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:12.393425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:12.393429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:12.394134 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:12.394726 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:12.394731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:12.394734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:12.395042 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:12.395888 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:12.395893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:12.396062 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 04:10:12.396175 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 04:10:12.396504 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9440,"thread_start_us":9944,"threads_started":1}
I20260807 04:10:12.398180 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:12.355955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:37311 (local address 127.18.193.60:44361)
0807 04:10:12.373906 (+ 17951us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:12.373912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:12.373934 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:10:12.373998 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:12.374003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:12.374083 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:12.374289 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:12.376258 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.377620 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.380918 (+ 3298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.381864 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.382008 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:12.388306 (+ 6298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:12.388315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:12.388320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:12.388367 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:12.393221 (+ 4854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.394313 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.394318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.394322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.394554 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.395194 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.395201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.395207 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.395711 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:12.395879 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:12.396371 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 04:10:12.396700 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17808,"thread_start_us":20113,"threads_started":1}
I20260807 04:10:12.398218 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:12.331899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.18.193.3:37081)
0807 04:10:12.353428 (+ 21529us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:12.353441 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:12.355244 (+ 1803us) server_negotiation.cc:407] Connection header received
0807 04:10:12.355324 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:12.355334 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:12.355602 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:12.355779 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:12.358210 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.359803 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.361767 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:12.362665 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:12.362765 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:12.371360 (+ 8595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:12.371383 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:12.371404 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:12.371461 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:12.393838 (+ 22377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.394853 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.394862 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.394866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.395133 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:12.395689 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:12.395694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:12.395697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:12.396305 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:12.396808 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:12.397140 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 04:10:12.397477 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21316,"thread_start_us":21152,"threads_started":1}
I20260807 04:10:12.399500 26842 heartbeater.cc:347] Connected to a master server at 127.18.193.60:44361
I20260807 04:10:12.399854 26842 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:12.400072 26136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:12.400506 26842 heartbeater.cc:511] Master 127.18.193.60:44361 requested a full tablet report, sending...
I20260807 04:10:12.402165 26505 ts_manager.cc:194] Registered new tserver with Master: 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879)
I20260807 04:10:12.491935 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 26007
I20260807 04:10:12.512523 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:37081
--local_ip_for_outbound_sockets=127.18.193.3
--tserver_master_addrs=127.18.193.62:35583
--webserver_port=35783
--webserver_interface=127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab
--principal=oryx/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:10:12.691030 26694 heartbeater.cc:503] Master 127.18.193.62:35583 was elected leader, sending a full tablet report...
W20260807 04:10:12.890214 26855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:12.890516 26855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:12.890615 26855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:12.898721 26855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:12.898883 26855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:12.899066 26855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:12.909628 26855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab
--principal=oryx/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:37081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=35783
--tserver_master_addrs=127.18.193.62:35583,127.18.193.61:46661,127.18.193.60:44361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:12.910977 26855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:12.912272 26855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:12.926509 26861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:12.926730 26864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:12.926936 26860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:12.927646 26855 server_base.cc:1034] running on GCE node
I20260807 04:10:12.928411 26855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:12.929472 26855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:12.930828 26855 hybrid_clock.cc:648] HybridClock initialized: now 1786075812930769 us; error 41 us; skew 500 ppm
I20260807 04:10:12.931264 26855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:12 dist-test-slave-8n49 krb5kdc[25612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:12.935077 26855 init.cc:377] Logged in from keytab as oryx/127.18.193.3@KRBTEST.COM (short username oryx)
I20260807 04:10:12.936743 26855 webserver.cc:460] Webserver started at http://127.18.193.3:35783/ using document root <none> and password file <none>
I20260807 04:10:12.937479 26855 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:12.937631 26855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:12.943281 26855 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:10:12.946164 26872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:12.947505 26855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 04:10:12.947695 26855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "e6a63168405b4888b2ce768da03f0709"
format_stamp: "Formatted at 2026-08-07 04:10:09 on dist-test-slave-8n49"
server_key: "8feddf41cbfa5fd888f11bcae8a67a10"
server_key_iv: "01a5b7a629f1c2754a807e60cedd23fc"
server_key_version: "encryptionkey@0"
I20260807 04:10:12.948774 26855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:13.008739 26855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:13.013502 26855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:13.013800 26855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:13.014868 26855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:13.016319 26855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:13.016438 26855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:13.016582 26855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:13.016806 26855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 04:10:13.049954 26855 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:37081
I20260807 04:10:13.050031 26989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:37081 every 8 connection(s)
I20260807 04:10:13.051553 26855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:13.061614 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 26855
Aug 07 04:10:13 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.60@KRBTEST.COM
Aug 07 04:10:13 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.61@KRBTEST.COM
I20260807 04:10:13.086951 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.054624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:58437 (local address 127.18.193.60:44361)
0807 04:10:13.055163 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.055170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.056716 (+ 1546us) server_negotiation.cc:407] Connection header received
0807 04:10:13.059166 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.059172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.059281 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.059374 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.061782 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.062936 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.064679 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.065101 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.065255 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.071837 (+ 6582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.071848 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.071855 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.071925 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.079030 (+ 7105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.080721 (+ 1691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.080730 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.080738 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.081114 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.082097 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.082104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.082107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.083321 (+ 1214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.083535 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.085746 (+ 2211us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.086113 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":249,"threads_started":1}
Aug 07 04:10:13 dist-test-slave-8n49 krb5kdc[25612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075812, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:13.096515 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.055524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.18.193.3:58437)
0807 04:10:13.056457 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.056581 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.058828 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.059751 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.059763 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.060636 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:13.061526 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.061560 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.063156 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.063159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.064480 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.064493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.065385 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.065393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.065796 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.066945 (+ 1149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:13.066985 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:13.071441 (+ 4456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:13.079364 (+ 7923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:13.079376 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:13.079402 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:13.080435 (+ 1033us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:13.081346 (+ 911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:13.081362 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:13.081367 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:13.081863 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:13.083672 (+ 1809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:13.083688 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:13.084422 (+ 734us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.084922 (+ 500us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.090848 (+ 5926us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":5,"spinlock_wait_cycles":68736,"thread_start_us":504,"threads_started":1}
I20260807 04:10:13.098999 26990 heartbeater.cc:347] Connected to a master server at 127.18.193.60:44361
I20260807 04:10:13.099478 26990 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:13.101176 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.055373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.18.193.3:41915)
0807 04:10:13.056457 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.056581 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.058828 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.059750 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.059758 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.060874 (+ 1116us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:13.061551 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.061560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.064693 (+ 3133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.064699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.066866 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.066889 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.077460 (+ 10571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.077471 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.077785 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.078774 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:13.078822 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:13.083633 (+ 4811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:13.095240 (+ 11607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:13.095245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:13.095248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:13.096098 (+ 850us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:13.097170 (+ 1072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:13.097179 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:13.097186 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:13.097494 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:13.099523 (+ 2029us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:13.099529 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:13.099807 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.099983 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.100393 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":109,"spinlock_wait_cycles":2048,"thread_start_us":445,"threads_started":1}
I20260807 04:10:13.102293 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.057105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.3:60133)
0807 04:10:13.057748 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.057772 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.058850 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.059877 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.059881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.060874 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:13.061544 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.061560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.071792 (+ 10232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.071798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.074184 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.074205 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.081684 (+ 7479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.081696 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.082094 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.084893 (+ 2799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:13.084917 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:13.090230 (+ 5313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:13.096692 (+ 6462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:13.096703 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:13.096708 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:13.097949 (+ 1241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:13.098428 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:13.098433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:13.098437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:13.099967 (+ 1530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:13.100930 (+ 963us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:13.100934 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:13.101177 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.101391 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.101725 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":129,"spinlock_wait_cycles":4224,"thread_start_us":332,"threads_started":1}
I20260807 04:10:13.102495 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.057268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:60133 (local address 127.18.193.62:35583)
0807 04:10:13.059558 (+ 2290us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.059566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.059585 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:10:13.059643 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.059648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.059728 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.059838 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.068741 (+ 8903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.069871 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.074399 (+ 4528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.075639 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.075835 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.090590 (+ 14755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.090599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.090604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.090669 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.096253 (+ 5584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.098087 (+ 1834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.098094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.098098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.098307 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.100126 (+ 1819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.100132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.100136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.100621 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.100820 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.101595 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.101872 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2145,"thread_start_us":2032,"threads_started":1}
I20260807 04:10:13.103472 26991 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35583
I20260807 04:10:13.103694 26991 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:13.104461 26995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.054839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:41915 (local address 127.18.193.61:46661)
0807 04:10:13.055510 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.055516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.057650 (+ 2134us) server_negotiation.cc:407] Connection header received
0807 04:10:13.059272 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.059282 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.059380 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.059488 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.061812 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.062936 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.068697 (+ 5761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.070370 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.076904 (+ 6534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.083951 (+ 7047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.083979 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.083987 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.084048 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.095068 (+ 11020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.096740 (+ 1672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.096746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.096749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.096995 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.098979 (+ 1984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.098984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.098986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.099330 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.099537 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.102726 (+ 3189us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.103731 (+ 1005us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":420,"server-negotiator.queue_time_us":344,"thread_start_us":696,"threads_started":1}
I20260807 04:10:13.105778 26990 heartbeater.cc:511] Master 127.18.193.60:44361 requested a full tablet report, sending...
I20260807 04:10:13.106478 26991 heartbeater.cc:511] Master 127.18.193.62:35583 requested a full tablet report, sending...
I20260807 04:10:13.106813 26992 heartbeater.cc:347] Connected to a master server at 127.18.193.61:46661
I20260807 04:10:13.106997 26992 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:13.107956 26992 heartbeater.cc:511] Master 127.18.193.61:46661 requested a full tablet report, sending...
I20260807 04:10:13.109059 25660 ts_manager.cc:194] Re-registered known tserver with Master: e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081)
I20260807 04:10:13.109490 26297 ts_manager.cc:194] Registered new tserver with Master: e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081)
I20260807 04:10:13.109558 26505 ts_manager.cc:194] Registered new tserver with Master: e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081)
I20260807 04:10:13.110603 25660 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:60133
I20260807 04:10:13.121135 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.065797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.18.193.61:46661)
0807 04:10:13.089355 (+ 23558us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.089363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.089390 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:10:13.089472 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.089480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.089603 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.092715 (+ 3112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.094441 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.096392 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.100877 (+ 4485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.103603 (+ 2726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.103756 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.110480 (+ 6724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.110488 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.110494 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.110555 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.116222 (+ 5667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.117281 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.117287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.117305 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.117540 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.118290 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.118295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.118299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.118836 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.119209 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.119950 (+ 741us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.120239 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":23356,"thread_start_us":23194,"threads_started":1}
I20260807 04:10:13.135079 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.098752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.18.193.60:44361)
0807 04:10:13.099828 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.099835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.111590 (+ 11755us) server_negotiation.cc:407] Connection header received
0807 04:10:13.111953 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.111959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.112059 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.112134 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.116165 (+ 4031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.117452 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.119502 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.120237 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.120430 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.126686 (+ 6256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.126693 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.126698 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.126740 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.130955 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.132120 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.132125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.132129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.132384 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.133113 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.133119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.133122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.133548 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.133782 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.134510 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.134737 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":924}
I20260807 04:10:13.135126 27007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.070542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.18.193.62:35583)
0807 04:10:13.088901 (+ 18359us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.088910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.094302 (+ 5392us) server_negotiation.cc:407] Connection header received
0807 04:10:13.094613 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.094621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.094715 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.094951 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.112061 (+ 17110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.114127 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.117229 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.118294 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.118461 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.126626 (+ 8165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.126634 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.126640 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.126701 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.130958 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.132120 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.132125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.132129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.132339 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.133113 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.133119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.133122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.133527 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.133782 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.134470 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.134719 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18194,"thread_start_us":17923,"threads_started":1}
I20260807 04:10:13.139842 25660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:10:13.147199 25660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:13.226090 27024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.173759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.18.193.2:33879)
0807 04:10:13.207962 (+ 34203us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.207977 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.208400 (+ 423us) server_negotiation.cc:407] Connection header received
0807 04:10:13.208607 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.208622 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.209070 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.209404 (+ 334us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:13.214215 (+ 4811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.216492 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.222940 (+ 6448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.224687 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.224839 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.225084 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.225513 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.225707 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33068,"thread_start_us":31985,"threads_started":1}
I20260807 04:10:13.226089 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.172417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:33879 (local address 127.0.0.1:57150)
0807 04:10:13.197810 (+ 25393us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.197843 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.198209 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.210639 (+ 12430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.210649 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.210702 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:13.211420 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.211440 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.216856 (+ 5416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.216867 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.221748 (+ 4881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.221758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.225061 (+ 3303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.225068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.225273 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.225310 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.225419 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.225564 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25160,"thread_start_us":46245,"threads_started":1}
I20260807 04:10:13.228852 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.172874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:45067 (local address 127.0.0.1:40836)
0807 04:10:13.217982 (+ 45108us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.218007 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.218226 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.219826 (+ 1600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.219830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.219861 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:13.220501 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.220514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.222676 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.222680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.224450 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.224463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.227248 (+ 2785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.227256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.227926 (+ 670us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.227974 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.228100 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.228259 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44745,"thread_start_us":25887,"threads_started":1}
I20260807 04:10:13.228950 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.173090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.18.193.1:45067)
0807 04:10:13.218530 (+ 45440us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.218544 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.218589 (+ 45us) server_negotiation.cc:407] Connection header received
0807 04:10:13.218716 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.218725 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.219127 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.219504 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:13.220783 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.222527 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.224675 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.226670 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.226815 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.226975 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.228149 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.228390 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45115,"thread_start_us":44879,"threads_started":1}
I20260807 04:10:13.230948 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 49bb5fbf3ffd441881d90fd443cc891b (DEFAULT_TABLE table=test-table [id=08229d51f34a4796a835ab3a8fa28f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:13.232494 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49bb5fbf3ffd441881d90fd443cc891b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:13.232415 26627 tablet_service.cc:1467] Processing CreateTablet for tablet 49bb5fbf3ffd441881d90fd443cc891b (DEFAULT_TABLE table=test-table [id=08229d51f34a4796a835ab3a8fa28f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:13.234227 26627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49bb5fbf3ffd441881d90fd443cc891b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:13.237998 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.207311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:37081 (local address 127.0.0.1:60872)
0807 04:10:13.219276 (+ 11965us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.219418 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.219717 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.220720 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.220724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.220750 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:13.221456 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.221470 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.225884 (+ 4414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.225889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.234109 (+ 8220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.234127 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.236692 (+ 2565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.236700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.237035 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.237112 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.237240 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.237433 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11799,"thread_start_us":9733,"threads_started":1}
I20260807 04:10:13.238016 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.200217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.18.193.3:37081)
0807 04:10:13.214073 (+ 13856us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.214088 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.219552 (+ 5464us) server_negotiation.cc:407] Connection header received
0807 04:10:13.219944 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.219952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.220190 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.220363 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:13.221735 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.225696 (+ 3961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.234416 (+ 8720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.236349 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.236507 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.236781 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.237340 (+ 559us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.237557 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13290,"thread_start_us":13357,"threads_started":1}
I20260807 04:10:13.242338 26924 tablet_service.cc:1467] Processing CreateTablet for tablet 49bb5fbf3ffd441881d90fd443cc891b (DEFAULT_TABLE table=test-table [id=08229d51f34a4796a835ab3a8fa28f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:13.243666 26924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49bb5fbf3ffd441881d90fd443cc891b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:13.252386 27027 tablet_bootstrap.cc:492] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Bootstrap starting.
I20260807 04:10:13.254873 27029 tablet_bootstrap.cc:492] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Bootstrap starting.
I20260807 04:10:13.258508 27027 tablet_bootstrap.cc:654] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:13.258558 27029 tablet_bootstrap.cc:654] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:13.259788 27029 log.cc:824] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:13.260409 27027 log.cc:824] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:13.262650 27032 tablet_bootstrap.cc:492] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Bootstrap starting.
I20260807 04:10:13.264166 27029 tablet_bootstrap.cc:492] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: No bootstrap required, opened a new log
I20260807 04:10:13.264464 27029 ts_tablet_manager.cc:1397] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260807 04:10:13.266077 27032 tablet_bootstrap.cc:654] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:13.267264 27032 log.cc:824] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:13.269137 27027 tablet_bootstrap.cc:492] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: No bootstrap required, opened a new log
I20260807 04:10:13.269423 27027 ts_tablet_manager.cc:1397] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.000s
I20260807 04:10:13.274284 27029 raft_consensus.cc:348] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.274827 27029 raft_consensus.cc:374] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:13.275022 27029 raft_consensus.cc:729] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0154e8ceb0ad44a9a8dc93e04b2f0c16, State: Initialized, Role: FOLLOWER
I20260807 04:10:13.275741 27029 consensus_queue.cc:260] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.276788 27032 tablet_bootstrap.cc:492] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: No bootstrap required, opened a new log
I20260807 04:10:13.277233 27032 ts_tablet_manager.cc:1397] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260807 04:10:13.278030 27029 ts_tablet_manager.cc:1428] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260807 04:10:13.280314 27027 raft_consensus.cc:348] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.280874 27027 raft_consensus.cc:374] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:13.280979 27027 raft_consensus.cc:729] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 474dd01821e746dfbdcf7fd31daaa13a, State: Initialized, Role: FOLLOWER
I20260807 04:10:13.281647 27027 consensus_queue.cc:260] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.286363 27027 ts_tablet_manager.cc:1428] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Time spent starting tablet: real 0.017s user 0.005s sys 0.012s
I20260807 04:10:13.286698 26843 heartbeater.cc:503] Master 127.18.193.62:35583 was elected leader, sending a full tablet report...
I20260807 04:10:13.291155 27032 raft_consensus.cc:348] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.291697 27032 raft_consensus.cc:374] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:13.291841 27032 raft_consensus.cc:729] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6a63168405b4888b2ce768da03f0709, State: Initialized, Role: FOLLOWER
I20260807 04:10:13.292522 27032 consensus_queue.cc:260] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.293898 26991 heartbeater.cc:503] Master 127.18.193.62:35583 was elected leader, sending a full tablet report...
I20260807 04:10:13.296520 27032 ts_tablet_manager.cc:1428] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
W20260807 04:10:13.304978 26993 tablet.cc:2369] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:10:13.331254 26845 tablet.cc:2369] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:10:13.347280 27036 raft_consensus.cc:482] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:13.347654 27036 raft_consensus.cc:504] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.349592 27036 leader_election.cc:288] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879), 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067)
I20260807 04:10:13.356858 27024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.351015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:53925 (local address 127.18.193.2:33879)
0807 04:10:13.351300 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.351306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.351327 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:10:13.351405 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.351411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.351499 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.351624 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:13.352034 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.353040 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.354775 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.355611 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.355784 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.355860 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.356320 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.356464 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:10:13.356860 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.349821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:33879 (local address 127.18.193.3:53925)
0807 04:10:13.350243 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.350280 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.350412 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.351642 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.351646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.351671 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:13.351890 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.351899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.353193 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.353199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.354611 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.354623 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.355800 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.355805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.356115 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.356166 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.356289 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.356471 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":35}
I20260807 04:10:13.356840 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.350298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:48085 (local address 127.18.193.1:45067)
0807 04:10:13.350555 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.350562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.350583 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:10:13.350897 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.350902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.350984 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.351095 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:13.351821 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.352830 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.354614 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.355612 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.355800 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.355877 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.356319 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.356479 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:10:13.356915 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.350113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:45067 (local address 127.18.193.3:48085)
0807 04:10:13.350424 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:13.350444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:13.350575 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:13.351300 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:13.351304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:13.351334 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:13.351625 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.351635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.352962 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.352969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.354441 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:13.354453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.355789 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.355795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:13.356148 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.356172 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:10:13.356288 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 04:10:13.356532 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":60}
I20260807 04:10:13.358202 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bb5fbf3ffd441881d90fd443cc891b" candidate_uuid: "e6a63168405b4888b2ce768da03f0709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474dd01821e746dfbdcf7fd31daaa13a" is_pre_election: true
I20260807 04:10:13.358785 26795 raft_consensus.cc:2454] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6a63168405b4888b2ce768da03f0709 in term 0.
I20260807 04:10:13.358762 26647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bb5fbf3ffd441881d90fd443cc891b" candidate_uuid: "e6a63168405b4888b2ce768da03f0709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" is_pre_election: true
I20260807 04:10:13.359280 26647 raft_consensus.cc:2454] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6a63168405b4888b2ce768da03f0709 in term 0.
I20260807 04:10:13.359576 26876 leader_election.cc:302] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 474dd01821e746dfbdcf7fd31daaa13a, e6a63168405b4888b2ce768da03f0709; no voters:
I20260807 04:10:13.360024 27036 raft_consensus.cc:2781] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:13.360180 27036 raft_consensus.cc:482] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:13.360277 27036 raft_consensus.cc:3037] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:13.362771 27036 raft_consensus.cc:504] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.363456 27036 leader_election.cc:288] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [CANDIDATE]: Term 1 election: Requested vote from peers 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879), 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067)
I20260807 04:10:13.363991 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bb5fbf3ffd441881d90fd443cc891b" candidate_uuid: "e6a63168405b4888b2ce768da03f0709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474dd01821e746dfbdcf7fd31daaa13a"
I20260807 04:10:13.364012 26647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bb5fbf3ffd441881d90fd443cc891b" candidate_uuid: "e6a63168405b4888b2ce768da03f0709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16"
I20260807 04:10:13.364197 26795 raft_consensus.cc:3037] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:13.364216 26647 raft_consensus.cc:3037] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:13.367154 26647 raft_consensus.cc:2454] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6a63168405b4888b2ce768da03f0709 in term 1.
I20260807 04:10:13.367198 26795 raft_consensus.cc:2454] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6a63168405b4888b2ce768da03f0709 in term 1.
I20260807 04:10:13.367849 26876 leader_election.cc:302] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0154e8ceb0ad44a9a8dc93e04b2f0c16, 474dd01821e746dfbdcf7fd31daaa13a, e6a63168405b4888b2ce768da03f0709; no voters:
I20260807 04:10:13.368363 27036 raft_consensus.cc:2781] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:13.369292 27036 raft_consensus.cc:686] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 1 LEADER]: Becoming Leader. State: Replica: e6a63168405b4888b2ce768da03f0709, State: Running, Role: LEADER
I20260807 04:10:13.370458 27036 consensus_queue.cc:237] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:13.379755 25659 catalog_manager.cc:5495] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6a63168405b4888b2ce768da03f0709 (127.18.193.3). New cstate: current_term: 1 leader_uuid: "e6a63168405b4888b2ce768da03f0709" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } health_report { overall_health: HEALTHY } } }
W20260807 04:10:13.394757 26696 tablet.cc:2369] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:10:13.429746 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.422183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.18.193.3:37081)
0807 04:10:13.422546 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.422553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.422728 (+ 175us) server_negotiation.cc:407] Connection header received
0807 04:10:13.423002 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.423008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.423109 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.423281 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:13.423797 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.424788 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.426612 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.427218 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.427330 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.428308 (+ 978us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:13.428948 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:13.429104 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.429251 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.429328 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 04:10:13.440477 26647 raft_consensus.cc:1261] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 FOLLOWER]: Refusing update from remote peer e6a63168405b4888b2ce768da03f0709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:13.440448 26795 raft_consensus.cc:1261] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Refusing update from remote peer e6a63168405b4888b2ce768da03f0709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:13.441753 27037 consensus_queue.cc:1035] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:13.442319 27036 consensus_queue.cc:1035] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:13.460737 27040 mvcc.cc:205] Tried to move back new op lower bound from 7315766531839619072 to 7315766531572850688. Current Snapshot: MvccSnapshot[applied={T|T < 7315766531839619072}]
I20260807 04:10:13.462939 27041 mvcc.cc:205] Tried to move back new op lower bound from 7315766531839619072 to 7315766531572850688. Current Snapshot: MvccSnapshot[applied={T|T < 7315766531839619072}]
I20260807 04:10:13.477324 27042 mvcc.cc:205] Tried to move back new op lower bound from 7315766531839619072 to 7315766531572850688. Current Snapshot: MvccSnapshot[applied={T|T < 7315766531839619072}]
I20260807 04:10:13.488714 25659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:13.489108 25659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:13.497112 25659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d3b1030fdf24283adc4bb0f87f8dfc8: Sending DeleteTablet for 3 replicas of tablet 49bb5fbf3ffd441881d90fd443cc891b
I20260807 04:10:13.498888 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 49bb5fbf3ffd441881d90fd443cc891b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:13 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:57150
I20260807 04:10:13.499996 26627 tablet_service.cc:1514] Processing DeleteTablet for tablet 49bb5fbf3ffd441881d90fd443cc891b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:13 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:40836
I20260807 04:10:13.500336 27051 tablet_replica.cc:330] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: stopping tablet replica
I20260807 04:10:13.500975 27051 raft_consensus.cc:2229] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:13.501410 26924 tablet_service.cc:1514] Processing DeleteTablet for tablet 49bb5fbf3ffd441881d90fd443cc891b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:13 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:60872
I20260807 04:10:13.501789 27051 raft_consensus.cc:2258] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:13.504541 27051 ts_tablet_manager.cc:1905] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:13.505316 27052 tablet_replica.cc:330] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: stopping tablet replica
I20260807 04:10:13.505900 27052 raft_consensus.cc:2229] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:13.506829 27052 raft_consensus.cc:2258] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:13.509761 27052 ts_tablet_manager.cc:1905] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:13.510459 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 25625
I20260807 04:10:13.511976 25693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:13.513070 27053 tablet_replica.cc:330] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: stopping tablet replica
I20260807 04:10:13.513782 27053 raft_consensus.cc:2229] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:10:13.514722 27053 raft_consensus.cc:2258] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:13.517721 27053 ts_tablet_manager.cc:1905] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:13.519382 27051 ts_tablet_manager.cc:1918] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:13.519671 27051 log.cc:1196] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/49bb5fbf3ffd441881d90fd443cc891b
I20260807 04:10:13.520799 27051 ts_tablet_manager.cc:1939] T 49bb5fbf3ffd441881d90fd443cc891b P 474dd01821e746dfbdcf7fd31daaa13a: Deleting consensus metadata
I20260807 04:10:13.531473 27052 ts_tablet_manager.cc:1918] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:13.532017 27052 log.cc:1196] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/49bb5fbf3ffd441881d90fd443cc891b
I20260807 04:10:13.533823 27052 ts_tablet_manager.cc:1939] T 49bb5fbf3ffd441881d90fd443cc891b P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Deleting consensus metadata
I20260807 04:10:13.540915 27053 ts_tablet_manager.cc:1918] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:13.541218 27053 log.cc:1196] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/49bb5fbf3ffd441881d90fd443cc891b
I20260807 04:10:13.542168 27053 ts_tablet_manager.cc:1939] T 49bb5fbf3ffd441881d90fd443cc891b P e6a63168405b4888b2ce768da03f0709: Deleting consensus metadata
I20260807 04:10:13.545827 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.507630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.18.193.61:46661)
0807 04:10:13.508445 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.508454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.509416 (+ 962us) server_negotiation.cc:407] Connection header received
0807 04:10:13.511210 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.511220 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.511348 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.511453 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.523278 (+ 11825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.524872 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.528095 (+ 3223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.529110 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.529222 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.535829 (+ 6607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.535840 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.535848 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.535902 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.540969 (+ 5067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.542268 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.542274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.542277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.542480 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.543014 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.543019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.543022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.543476 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.543809 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.545284 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.545500 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634}
I20260807 04:10:13.565580 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.509939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.18.193.60:44361)
0807 04:10:13.510212 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.510219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.547289 (+ 37070us) server_negotiation.cc:407] Connection header received
0807 04:10:13.547555 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.547566 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.547639 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.547755 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.549735 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.550875 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.552770 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.553611 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.553764 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.557737 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.557742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.557745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.557792 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.561695 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.562616 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.562622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.562626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.562827 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.563315 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.563320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.563324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.563727 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.564251 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.564614 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.564832 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:10:13.701336 25643 catalog_manager.cc:4841] TS e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081): tablet 49bb5fbf3ffd441881d90fd443cc891b (table test-table [id=08229d51f34a4796a835ab3a8fa28f00]) successfully deleted
I20260807 04:10:13.701810 25643 catalog_manager.cc:4841] TS 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067): tablet 49bb5fbf3ffd441881d90fd443cc891b (table test-table [id=08229d51f34a4796a835ab3a8fa28f00]) successfully deleted
I20260807 04:10:13.702436 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 25625
I20260807 04:10:13.751863 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.734631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.18.193.60:44361)
0807 04:10:13.734916 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.734925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.735724 (+ 799us) server_negotiation.cc:407] Connection header received
0807 04:10:13.735920 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.735924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.735992 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.736116 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.738719 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.739816 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.741377 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.741940 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.742064 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.745518 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.745527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.745532 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.745578 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.748695 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.749460 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.749467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.749469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.749628 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.750233 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.750238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.750242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.750631 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.750897 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.751312 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.751552 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:10:13.753928 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:13.733566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39924 (local address 127.18.193.61:46661)
0807 04:10:13.733770 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:13.733777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:13.735013 (+ 1236us) server_negotiation.cc:407] Connection header received
0807 04:10:13.736115 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:13.736122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:13.736202 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:13.736310 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:13.738233 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.739527 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.741676 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:13.742300 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:13.742446 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:13.746243 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:13.746251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:13.746257 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:13.746305 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:13.750280 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.751475 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.751481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.751485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.751650 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:13.752127 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:13.752133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:13.752136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:13.752593 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:13.752820 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:13.753409 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 04:10:13.753664 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:10:14.473171 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:14.472071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.2:50065)
0807 04:10:14.472765 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:14.472942 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":369,"threads_started":1}
W20260807 04:10:14.473757 26843 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:35583 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
I20260807 04:10:14.478986 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:14.478093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.3:56405)
0807 04:10:14.478620 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:14.478781 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":228,"threads_started":1}
W20260807 04:10:14.479368 26991 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:35583 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
I20260807 04:10:14.482733 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:14.482043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:58705)
0807 04:10:14.482457 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:14.482583 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":203,"threads_started":1}
W20260807 04:10:14.483146 26694 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:35583 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
I20260807 04:10:14.997018 27067 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0d3b1030fdf24283adc4bb0f87f8dfc8)
I20260807 04:10:14.997253 27068 raft_consensus.cc:482] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0d3b1030fdf24283adc4bb0f87f8dfc8)
I20260807 04:10:14.997253 27067 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } }
I20260807 04:10:14.997552 27068 raft_consensus.cc:504] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } }
I20260807 04:10:14.998173 27067 leader_election.cc:288] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583), 355f9a504f564ffd82691adf27dab598 (127.18.193.61:46661)
I20260807 04:10:14.999392 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:14.998241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51720)
0807 04:10:14.998879 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:14.999084 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":361,"threads_started":1}
W20260807 04:10:14.999683 26492 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
I20260807 04:10:15.001225 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:14.999890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51724)
0807 04:10:15.000612 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.000876 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":368,"threads_started":1}
I20260807 04:10:15.002283 27068 leader_election.cc:288] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583), b2ebdf88a8634958bc9c1a32301d99ee (127.18.193.60:44361)
I20260807 04:10:15.003538 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.002796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51726)
0807 04:10:15.003033 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.003301 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260807 04:10:15.003741 26492 leader_election.cc:334] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583): Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
W20260807 04:10:15.004279 26284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
I20260807 04:10:15.006140 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.005475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51742)
0807 04:10:15.005669 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.005925 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 04:10:15.006474 26284 leader_election.cc:334] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583): Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
I20260807 04:10:15.008066 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:14.998402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39928)
0807 04:10:14.999610 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:14.999627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:14.999838 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:15.000625 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:15.000633 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:15.000716 (+ 83us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:15.000949 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.000976 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.003080 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.003089 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.005625 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.005644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.007341 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.007346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.007496 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.007521 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:10:15.007608 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 04:10:15.007701 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"mutex_wait_us":3,"thread_start_us":995,"threads_started":1}
I20260807 04:10:15.008077 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:14.998510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.18.193.61:46661)
0807 04:10:14.999218 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:14.999225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:14.999718 (+ 493us) server_negotiation.cc:407] Connection header received
0807 04:10:15.000060 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.000069 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.000183 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.000314 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:15.001200 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.002829 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.005789 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.007167 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.007305 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.007409 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.007658 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.007798 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":359,"threads_started":1}
I20260807 04:10:15.008733 26308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "355f9a504f564ffd82691adf27dab598" is_pre_election: true
I20260807 04:10:15.009032 26308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2ebdf88a8634958bc9c1a32301d99ee in term 1.
I20260807 04:10:15.009541 26491 leader_election.cc:302] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 355f9a504f564ffd82691adf27dab598, b2ebdf88a8634958bc9c1a32301d99ee; no voters: 0d3b1030fdf24283adc4bb0f87f8dfc8
I20260807 04:10:15.009874 27067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 04:10:15.010027 27067 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0d3b1030fdf24283adc4bb0f87f8dfc8)
I20260807 04:10:15.010085 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.000868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:44361 (local address 127.0.0.1:44264)
0807 04:10:15.001507 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.001533 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:15.001683 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:15.003392 (+ 1709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:15.003398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:15.003425 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:15.003713 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.003729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.005550 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.005558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.007730 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.007744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.009253 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.009260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.009466 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.009490 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:10:15.009600 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 04:10:15.009736 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":1,"thread_start_us":3136,"threads_started":1}
I20260807 04:10:15.010156 27067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:10:15.010423 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.001274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.18.193.60:44361)
0807 04:10:15.003014 (+ 1740us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.003024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.003047 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:10:15.003103 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.003110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.003200 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.003867 (+ 667us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:15.004205 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.005418 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.007897 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.009060 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.009195 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.009298 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.009628 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.009755 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":1600,"thread_start_us":1455,"threads_started":1}
I20260807 04:10:15.010985 26516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "355f9a504f564ffd82691adf27dab598" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" is_pre_election: true
I20260807 04:10:15.013206 27067 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } }
I20260807 04:10:15.013947 26516 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 355f9a504f564ffd82691adf27dab598 in current term 2: Already voted for candidate b2ebdf88a8634958bc9c1a32301d99ee in this term.
I20260807 04:10:15.014106 27067 leader_election.cc:288] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 2 election: Requested vote from peers 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583), 355f9a504f564ffd82691adf27dab598 (127.18.193.61:46661)
I20260807 04:10:15.014596 26308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "355f9a504f564ffd82691adf27dab598"
I20260807 04:10:15.014699 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.014146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51758)
0807 04:10:15.014413 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.014536 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260807 04:10:15.014824 26308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:10:15.015158 26284 leader_election.cc:302] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 355f9a504f564ffd82691adf27dab598; no voters: 0d3b1030fdf24283adc4bb0f87f8dfc8, b2ebdf88a8634958bc9c1a32301d99ee
I20260807 04:10:15.015751 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.015352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51762)
0807 04:10:15.015585 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.015661 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 04:10:15.015972 26492 leader_election.cc:334] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583): Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
I20260807 04:10:15.017408 26308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2ebdf88a8634958bc9c1a32301d99ee in term 2.
I20260807 04:10:15.017679 27068 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:10:15.017843 26491 leader_election.cc:302] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 355f9a504f564ffd82691adf27dab598, b2ebdf88a8634958bc9c1a32301d99ee; no voters: 0d3b1030fdf24283adc4bb0f87f8dfc8
I20260807 04:10:15.018209 27067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:10:15.018733 27067 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [term 2 LEADER]: Becoming Leader. State: Replica: b2ebdf88a8634958bc9c1a32301d99ee, State: Running, Role: LEADER
I20260807 04:10:15.019119 27067 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } }
I20260807 04:10:15.020718 27078 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2ebdf88a8634958bc9c1a32301d99ee. Latest consensus state: current_term: 2 leader_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:15.021049 27078 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:15.021766 27080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:15.025818 27080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:15.026688 27080 catalog_manager.cc:1259] Loaded cluster ID: e74ee0415bb5441bbdd3cc3eccdb9151
I20260807 04:10:15.026757 27080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:15.027969 27080 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:15.028905 27080 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Loaded TSK: 0
I20260807 04:10:15.029790 27080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:10:15.044150 26505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:10:15.051381 26505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:15.056205 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.055652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51780)
0807 04:10:15.055919 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.056032 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 04:10:15.056162 26308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [term 2 FOLLOWER]: Refusing update from remote peer b2ebdf88a8634958bc9c1a32301d99ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 04:10:15.057076 27078 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 04:10:15.057224 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.056865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51796)
0807 04:10:15.057022 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.057093 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 04:10:15.057394 26492 consensus_peers.cc:487] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee -> Peer 0d3b1030fdf24283adc4bb0f87f8dfc8 (127.18.193.62:35583): Couldn't send request to peer 0d3b1030fdf24283adc4bb0f87f8dfc8. Status: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 04:10:15.059557 27068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2ebdf88a8634958bc9c1a32301d99ee. Latest consensus state: current_term: 2 leader_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:15.059796 27068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:10:15.061584 27067 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:15.062016 27067 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:15.062454 27082 mvcc.cc:205] Tried to move back new op lower bound from 7315766538463358976 to 7315766538321477632. Current Snapshot: MvccSnapshot[applied={T|T < 7315766538463358976}]
I20260807 04:10:15.062860 27068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0d3b1030fdf24283adc4bb0f87f8dfc8" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35583 } } peers { permanent_uuid: "355f9a504f564ffd82691adf27dab598" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 46661 } attrs { promote: false } } peers { permanent_uuid: "b2ebdf88a8634958bc9c1a32301d99ee" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 44361 } attrs { promote: false } } }
I20260807 04:10:15.063123 27068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355f9a504f564ffd82691adf27dab598 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:10:15.089169 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.069816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:33879 (local address 127.0.0.1:57162)
0807 04:10:15.070272 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.070291 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:15.070409 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:15.083349 (+ 12940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:15.083355 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:15.083384 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:15.083616 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.083628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.085315 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.085323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.087182 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.087195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.088298 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.088304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.088507 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.088550 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 04:10:15.088688 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 04:10:15.088833 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":69}
I20260807 04:10:15.089179 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.069964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.18.193.2:33879)
0807 04:10:15.082805 (+ 12841us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.082815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.082844 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:10:15.082948 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.082955 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.083063 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.083768 (+ 705us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:15.083938 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.085154 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.087355 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.088125 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.088273 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.088380 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.088735 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.088887 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12643,"thread_start_us":13197,"threads_started":1}
I20260807 04:10:15.090282 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 47f00de03e8f45a484731645fa53cd1f (DEFAULT_TABLE table=test-table [id=13b52ebfa84f46a996ebdcda420ca1e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:15.090792 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f00de03e8f45a484731645fa53cd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:15.090874 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.070547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.18.193.1:45067)
0807 04:10:15.083789 (+ 13242us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.083794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.083810 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:10:15.083870 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.083873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.083930 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.084059 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:15.084816 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.085949 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.088607 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.089825 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.089965 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.090077 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.090437 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.090576 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13134,"thread_start_us":13047,"threads_started":1}
I20260807 04:10:15.090893 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.070199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:45067 (local address 127.0.0.1:40844)
0807 04:10:15.070315 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.070336 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:15.070412 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:15.084076 (+ 13664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:15.084081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:15.084108 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:15.084626 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.084646 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.086097 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.086102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.088400 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.088416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.090051 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.090057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.090245 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.090287 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 04:10:15.090387 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 04:10:15.090543 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":38}
I20260807 04:10:15.091131 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.070916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:37081 (local address 127.0.0.1:60894)
0807 04:10:15.083409 (+ 12493us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.083433 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:15.083555 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:15.084325 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:15.084332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:15.084364 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:15.084818 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.084829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.086336 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.086343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.088444 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.088457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.090284 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.090289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.090478 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.090509 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:10:15.090584 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 04:10:15.090715 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12342,"mutex_wait_us":1,"spinlock_wait_cycles":28416,"thread_start_us":12129,"threads_started":1}
I20260807 04:10:15.091751 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.071092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.18.193.3:37081)
0807 04:10:15.083994 (+ 12902us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.084001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.084023 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:10:15.084096 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.084101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.084170 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.084277 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:15.084991 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.085841 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.088675 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.089855 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.090015 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.090140 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.091371 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.091505 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12759,"thread_start_us":13346,"threads_started":1}
I20260807 04:10:15.091717 26627 tablet_service.cc:1467] Processing CreateTablet for tablet 47f00de03e8f45a484731645fa53cd1f (DEFAULT_TABLE table=test-table [id=13b52ebfa84f46a996ebdcda420ca1e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:15.092141 26627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f00de03e8f45a484731645fa53cd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:15.092505 26924 tablet_service.cc:1467] Processing CreateTablet for tablet 47f00de03e8f45a484731645fa53cd1f (DEFAULT_TABLE table=test-table [id=13b52ebfa84f46a996ebdcda420ca1e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:15.092980 26924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f00de03e8f45a484731645fa53cd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:15.098174 27092 tablet_bootstrap.cc:492] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: Bootstrap starting.
I20260807 04:10:15.099994 27092 tablet_bootstrap.cc:654] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:15.101111 27094 tablet_bootstrap.cc:492] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Bootstrap starting.
I20260807 04:10:15.103812 27094 tablet_bootstrap.cc:654] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:15.105528 27093 tablet_bootstrap.cc:492] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: Bootstrap starting.
I20260807 04:10:15.106895 27092 tablet_bootstrap.cc:492] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: No bootstrap required, opened a new log
I20260807 04:10:15.107067 27092 ts_tablet_manager.cc:1397] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 04:10:15.107383 27093 tablet_bootstrap.cc:654] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:15.107517 27094 tablet_bootstrap.cc:492] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: No bootstrap required, opened a new log
I20260807 04:10:15.107663 27094 ts_tablet_manager.cc:1397] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 04:10:15.107885 27092 raft_consensus.cc:348] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.108104 27092 raft_consensus.cc:374] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:15.108207 27092 raft_consensus.cc:729] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6a63168405b4888b2ce768da03f0709, State: Initialized, Role: FOLLOWER
I20260807 04:10:15.108494 27092 consensus_queue.cc:260] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.108587 27094 raft_consensus.cc:348] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.108923 27094 raft_consensus.cc:374] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:15.109041 27094 raft_consensus.cc:729] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0154e8ceb0ad44a9a8dc93e04b2f0c16, State: Initialized, Role: FOLLOWER
I20260807 04:10:15.109371 27094 consensus_queue.cc:260] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.113073 27092 ts_tablet_manager.cc:1428] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 04:10:15.113618 27094 ts_tablet_manager.cc:1428] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 04:10:15.122062 27093 tablet_bootstrap.cc:492] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: No bootstrap required, opened a new log
I20260807 04:10:15.122241 27093 ts_tablet_manager.cc:1397] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.015s
I20260807 04:10:15.123260 27093 raft_consensus.cc:348] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.123577 27093 raft_consensus.cc:374] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:15.123627 26990 heartbeater.cc:503] Master 127.18.193.60:44361 was elected leader, sending a full tablet report...
I20260807 04:10:15.123682 27093 raft_consensus.cc:729] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 474dd01821e746dfbdcf7fd31daaa13a, State: Initialized, Role: FOLLOWER
I20260807 04:10:15.123908 27093 consensus_queue.cc:260] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.124379 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.114231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.3:54177)
0807 04:10:15.123443 (+ 9212us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.123678 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9067,"thread_start_us":8861,"threads_started":1}
I20260807 04:10:15.125168 26693 heartbeater.cc:503] Master 127.18.193.60:44361 was elected leader, sending a full tablet report...
I20260807 04:10:15.126936 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.116638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:39025)
0807 04:10:15.126389 (+ 9751us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.126590 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8215,"thread_start_us":8058,"threads_started":1}
I20260807 04:10:15.127126 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.126406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.3:40953)
0807 04:10:15.126780 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.126881 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276}
I20260807 04:10:15.127655 27093 ts_tablet_manager.cc:1428] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260807 04:10:15.129132 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.128183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:34657)
0807 04:10:15.128859 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.128944 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319}
I20260807 04:10:15.129181 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.127874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.2:53541)
0807 04:10:15.128609 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.128806 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":366,"threads_started":1}
I20260807 04:10:15.130191 26842 heartbeater.cc:503] Master 127.18.193.60:44361 was elected leader, sending a full tablet report...
I20260807 04:10:15.130297 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.129811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.2:43897)
0807 04:10:15.130041 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.130124 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 04:10:15.261348 27100 raft_consensus.cc:482] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:15.261646 27100 raft_consensus.cc:504] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.263429 27100 leader_election.cc:288] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879), e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081)
I20260807 04:10:15.273838 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.264038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:45495 (local address 127.18.193.3:37081)
0807 04:10:15.264356 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.264366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.265666 (+ 1300us) server_negotiation.cc:407] Connection header received
0807 04:10:15.265965 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.265973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.266077 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.266148 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:15.267016 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.268310 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.271314 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.272493 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.272819 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.272918 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.273359 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.273535 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260807 04:10:15.273921 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.263864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:36513 (local address 127.18.193.2:33879)
0807 04:10:15.264112 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.264121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.264144 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:10:15.264513 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.264521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.264675 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.264765 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:15.265678 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.267734 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.270709 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.272531 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.272712 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.272795 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.273488 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.273641 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:10:15.273981 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.263786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:37081 (local address 127.18.193.1:45495)
0807 04:10:15.265552 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.265572 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:15.265781 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:15.266373 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:15.266390 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:15.266420 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:15.266762 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.266774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.268512 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.268520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.271138 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.271158 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.272723 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.272731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.273071 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.273116 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 04:10:15.273327 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 04:10:15.273521 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1622,"mutex_wait_us":342,"thread_start_us":1184,"threads_started":1}
I20260807 04:10:15.274592 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.263792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:33879 (local address 127.18.193.1:36513)
0807 04:10:15.264024 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.264048 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:15.264224 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:15.265010 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:15.265018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:15.265054 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:15.265444 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.265472 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.267936 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.267941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.270501 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:15.270519 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.272974 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.272984 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:15.273239 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.273285 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 04:10:15.274169 (+ 884us) client_negotiation.cc:240] Negotiation successful
0807 04:10:15.274341 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":32}
I20260807 04:10:15.275208 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f00de03e8f45a484731645fa53cd1f" candidate_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474dd01821e746dfbdcf7fd31daaa13a" is_pre_election: true
I20260807 04:10:15.275224 26944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f00de03e8f45a484731645fa53cd1f" candidate_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a63168405b4888b2ce768da03f0709" is_pre_election: true
I20260807 04:10:15.275532 26795 raft_consensus.cc:2454] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0154e8ceb0ad44a9a8dc93e04b2f0c16 in term 0.
I20260807 04:10:15.275651 26944 raft_consensus.cc:2454] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0154e8ceb0ad44a9a8dc93e04b2f0c16 in term 0.
I20260807 04:10:15.276185 26581 leader_election.cc:302] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0154e8ceb0ad44a9a8dc93e04b2f0c16, 474dd01821e746dfbdcf7fd31daaa13a; no voters:
I20260807 04:10:15.276670 27100 raft_consensus.cc:2781] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:15.276805 27100 raft_consensus.cc:482] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:15.276907 27100 raft_consensus.cc:3037] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:15.279037 27100 raft_consensus.cc:504] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.279719 27100 leader_election.cc:288] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [CANDIDATE]: Term 1 election: Requested vote from peers 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879), e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081)
I20260807 04:10:15.280174 26944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f00de03e8f45a484731645fa53cd1f" candidate_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6a63168405b4888b2ce768da03f0709"
I20260807 04:10:15.280171 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f00de03e8f45a484731645fa53cd1f" candidate_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "474dd01821e746dfbdcf7fd31daaa13a"
I20260807 04:10:15.280380 26944 raft_consensus.cc:3037] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:15.280400 26795 raft_consensus.cc:3037] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:15.283427 26795 raft_consensus.cc:2454] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0154e8ceb0ad44a9a8dc93e04b2f0c16 in term 1.
I20260807 04:10:15.283425 26944 raft_consensus.cc:2454] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0154e8ceb0ad44a9a8dc93e04b2f0c16 in term 1.
I20260807 04:10:15.283831 26579 leader_election.cc:302] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0154e8ceb0ad44a9a8dc93e04b2f0c16, e6a63168405b4888b2ce768da03f0709; no voters:
I20260807 04:10:15.284161 27100 raft_consensus.cc:2781] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:15.284664 27100 raft_consensus.cc:686] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 LEADER]: Becoming Leader. State: Replica: 0154e8ceb0ad44a9a8dc93e04b2f0c16, State: Running, Role: LEADER
I20260807 04:10:15.285290 27100 consensus_queue.cc:237] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } }
I20260807 04:10:15.285780 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.285331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:54673)
0807 04:10:15.285506 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.285629 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 04:10:15.289389 26504 catalog_manager.cc:5495] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0154e8ceb0ad44a9a8dc93e04b2f0c16" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 45067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 } health_report { overall_health: UNKNOWN } } }
I20260807 04:10:15.325573 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.319564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.18.193.1:45067)
0807 04:10:15.319779 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.319783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.319937 (+ 154us) server_negotiation.cc:407] Connection header received
0807 04:10:15.320074 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.320078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.320161 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.320293 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:15.320856 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.321575 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.323034 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.323632 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.323744 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.324174 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:15.324867 (+ 693us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:15.324995 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.325176 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.325265 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:10:15.330264 26795 raft_consensus.cc:1261] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Refusing update from remote peer 0154e8ceb0ad44a9a8dc93e04b2f0c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:15.330770 26944 raft_consensus.cc:1261] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 1 FOLLOWER]: Refusing update from remote peer 0154e8ceb0ad44a9a8dc93e04b2f0c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:15.331003 27106 consensus_queue.cc:1035] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474dd01821e746dfbdcf7fd31daaa13a" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 33879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:15.331573 27100 consensus_queue.cc:1035] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6a63168405b4888b2ce768da03f0709" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 37081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:15.335218 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.334667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.3:41023)
0807 04:10:15.334889 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.335029 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 04:10:15.336843 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.336120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.2:52051)
0807 04:10:15.336409 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.336572 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260807 04:10:15.347047 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.346493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.3:33353)
0807 04:10:15.346732 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.346835 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260807 04:10:15.347149 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.346155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.2:45441)
0807 04:10:15.346905 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.346993 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641}
I20260807 04:10:15.348945 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.348351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:33977)
0807 04:10:15.348624 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.348719 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260807 04:10:15.350147 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.349673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.18.193.1:43953)
0807 04:10:15.349892 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.349986 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 04:10:15.360145 26505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:15.360399 26505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:15.367301 26505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee: Sending DeleteTablet for 3 replicas of tablet 47f00de03e8f45a484731645fa53cd1f
I20260807 04:10:15.368436 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 47f00de03e8f45a484731645fa53cd1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:15 UTC) from {username='oryx', principal='oryx/127.18.193.60@KRBTEST.COM'} at 127.0.0.1:57162
I20260807 04:10:15.368881 26627 tablet_service.cc:1514] Processing DeleteTablet for tablet 47f00de03e8f45a484731645fa53cd1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:15 UTC) from {username='oryx', principal='oryx/127.18.193.60@KRBTEST.COM'} at 127.0.0.1:40844
I20260807 04:10:15.369357 26924 tablet_service.cc:1514] Processing DeleteTablet for tablet 47f00de03e8f45a484731645fa53cd1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:15 UTC) from {username='oryx', principal='oryx/127.18.193.60@KRBTEST.COM'} at 127.0.0.1:60894
I20260807 04:10:15.373867 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 26480
I20260807 04:10:15.377842 27118 tablet_replica.cc:330] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: stopping tablet replica
I20260807 04:10:15.378118 27118 raft_consensus.cc:2229] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:15.378465 27118 raft_consensus.cc:2258] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:15.385106 27120 tablet_replica.cc:330] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: stopping tablet replica
I20260807 04:10:15.385412 27120 raft_consensus.cc:2229] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:15.385847 27120 raft_consensus.cc:2258] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:15.386471 27119 tablet_replica.cc:330] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: stopping tablet replica
I20260807 04:10:15.386936 27119 raft_consensus.cc:2229] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:10:15.387688 27119 raft_consensus.cc:2258] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:15.387771 27120 ts_tablet_manager.cc:1905] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:15.388051 27118 ts_tablet_manager.cc:1905] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:15.388865 27119 ts_tablet_manager.cc:1905] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:15.395044 27120 ts_tablet_manager.cc:1918] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:15.395404 27120 log.cc:1196] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/47f00de03e8f45a484731645fa53cd1f
I20260807 04:10:15.395898 27120 ts_tablet_manager.cc:1939] T 47f00de03e8f45a484731645fa53cd1f P e6a63168405b4888b2ce768da03f0709: Deleting consensus metadata
I20260807 04:10:15.397073 27119 ts_tablet_manager.cc:1918] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:15.397240 26490 catalog_manager.cc:4841] TS e6a63168405b4888b2ce768da03f0709 (127.18.193.3:37081): tablet 47f00de03e8f45a484731645fa53cd1f (table test-table [id=13b52ebfa84f46a996ebdcda420ca1e2]) successfully deleted
I20260807 04:10:15.397467 27119 log.cc:1196] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/47f00de03e8f45a484731645fa53cd1f
I20260807 04:10:15.398128 27119 ts_tablet_manager.cc:1939] T 47f00de03e8f45a484731645fa53cd1f P 0154e8ceb0ad44a9a8dc93e04b2f0c16: Deleting consensus metadata
I20260807 04:10:15.399483 26490 catalog_manager.cc:4841] TS 0154e8ceb0ad44a9a8dc93e04b2f0c16 (127.18.193.1:45067): tablet 47f00de03e8f45a484731645fa53cd1f (table test-table [id=13b52ebfa84f46a996ebdcda420ca1e2]) successfully deleted
I20260807 04:10:15.402650 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.376151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.18.193.1:45067)
0807 04:10:15.376321 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.376327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.377254 (+ 927us) server_negotiation.cc:407] Connection header received
0807 04:10:15.377460 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.377468 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.377590 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.377693 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:15.382296 (+ 4603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.383741 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.386427 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.387237 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.387349 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.395331 (+ 7982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:15.395350 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:15.395371 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:15.395429 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:15.399204 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:15.400099 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:15.400108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:15.400112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:15.400298 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:15.400853 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:15.400858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:15.400862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:15.401322 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:15.401659 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.402040 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.402272 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:10:15.403191 27118 ts_tablet_manager.cc:1918] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:15.403360 27118 log.cc:1196] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/47f00de03e8f45a484731645fa53cd1f
I20260807 04:10:15.403430 26688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:15.403810 27118 ts_tablet_manager.cc:1939] T 47f00de03e8f45a484731645fa53cd1f P 474dd01821e746dfbdcf7fd31daaa13a: Deleting consensus metadata
I20260807 04:10:15.404846 26492 catalog_manager.cc:4841] TS 474dd01821e746dfbdcf7fd31daaa13a (127.18.193.2:33879): tablet 47f00de03e8f45a484731645fa53cd1f (table test-table [id=13b52ebfa84f46a996ebdcda420ca1e2]) successfully deleted
I20260807 04:10:15.513023 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 26480
I20260807 04:10:15.521938 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.521476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51798)
0807 04:10:15.521681 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.521781 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 04:10:15.523120 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.522736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51810)
0807 04:10:15.522896 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.522979 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 04:10:15.534209 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 26706
I20260807 04:10:15.548269 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.534995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.18.193.2:33879)
0807 04:10:15.535186 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.535194 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.535220 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:10:15.535460 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.535466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.535556 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.535665 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:15.537102 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.537862 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.539110 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.539504 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.539638 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.542361 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:15.542376 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:15.542394 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:15.542430 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:15.545404 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:15.546148 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:15.546155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:15.546157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:15.546291 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:15.546686 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:15.546691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:15.546694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:15.547077 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:15.547398 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.547701 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.547931 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:10:15.549044 26837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:15.645375 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 26706
I20260807 04:10:15.665107 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 26855
I20260807 04:10:15.678828 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.665831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.18.193.3:37081)
0807 04:10:15.666064 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:15.666073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:15.666165 (+ 92us) server_negotiation.cc:407] Connection header received
0807 04:10:15.666386 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:15.666390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:15.666459 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:15.666579 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:15.668006 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.668809 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.670027 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:15.670422 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:15.670558 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:15.673165 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:15.673180 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:15.673197 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:15.673226 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:15.675935 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:15.676712 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:15.676727 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:15.676729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:15.676870 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:15.677253 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:15.677258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:15.677261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:15.677622 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:15.677937 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:15.678273 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 04:10:15.678506 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 04:10:15.679593 26985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:15.802475 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 26855
I20260807 04:10:15.827714 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 26148
I20260807 04:10:15.828944 26332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:15.948666 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 26148
W20260807 04:10:15.972582 26491 connection.cc:535] client connection to 127.18.193.61:46661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 04:10:15.972946 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 26254
I20260807 04:10:15.973973 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:15.973465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:46661 (local address 127.0.0.1:39936)
0807 04:10:15.973684 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:15.973785 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:46661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 04:10:15.974133 26537 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 04:10:15.974177 26491 consensus_peers.cc:487] T 00000000000000000000000000000000 P b2ebdf88a8634958bc9c1a32301d99ee -> Peer 355f9a504f564ffd82691adf27dab598 (127.18.193.61:46661): Couldn't send request to peer 355f9a504f564ffd82691adf27dab598. Status: Network error: Client connection negotiation failed: client connection to 127.18.193.61:46661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 04:10:16.102448 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:16.101890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35583 (local address 127.0.0.1:51820)
0807 04:10:16.102088 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:16.102204 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 04:10:16.102447 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 26254
2026-08-07T04:10:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10544 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:10:16 dist-test-slave-8n49 krb5kdc[27129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:10:16 dist-test-slave-8n49 krb5kdc[27129](info): set up 2 sockets
Aug 07 04:10:16 dist-test-slave-8n49 krb5kdc[27129](info): commencing operation
krb5kdc: starting...
W20260807 04:10:17.295948 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:10:17 dist-test-slave-8n49 krb5kdc[27129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:10:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab.
Entry for principal oryx/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab.
I20260807 04:10:17.569026 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44645
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab
--principal=oryx/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:44645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:17.912276 27142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:17.912703 27142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:17.912878 27142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:17.920329 27142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:17.920516 27142 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 04:10:17.920642 27142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:17.920764 27142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:17.920868 27142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:17.920971 27142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:17.931460 27142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:44645
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.62.keytab
--principal=oryx/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:17.932957 27142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:17.934335 27142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:17.946627 27147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:17.946627 27148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:17.946663 27151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:17.947626 27142 server_base.cc:1034] running on GCE node
I20260807 04:10:17.948508 27142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:17.950057 27142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:17.951303 27142 hybrid_clock.cc:648] HybridClock initialized: now 1786075817951231 us; error 56 us; skew 500 ppm
I20260807 04:10:17.951632 27142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:17 dist-test-slave-8n49 krb5kdc[27129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075817, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:17.954867 27142 init.cc:377] Logged in from keytab as oryx/127.18.193.62@KRBTEST.COM (short username oryx)
I20260807 04:10:17.956530 27142 webserver.cc:460] Webserver started at http://127.18.193.62:36461/ using document root <none> and password file <none>
I20260807 04:10:17.957338 27142 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:17.957479 27142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:17.957818 27142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:17.961036 27142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "a62a00f38026409ba35607725cba3a2e"
format_stamp: "Formatted at 2026-08-07 04:10:17 on dist-test-slave-8n49"
server_key: "49fb4f8222c1f6695d193a5e1c981fa2"
server_key_iv: "591aa3c54ead3b49ab53ea58a5967c31"
server_key_version: "encryptionkey@0"
I20260807 04:10:17.961875 27142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "a62a00f38026409ba35607725cba3a2e"
format_stamp: "Formatted at 2026-08-07 04:10:17 on dist-test-slave-8n49"
server_key: "49fb4f8222c1f6695d193a5e1c981fa2"
server_key_iv: "591aa3c54ead3b49ab53ea58a5967c31"
server_key_version: "encryptionkey@0"
I20260807 04:10:17.967231 27142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:10:17.970893 27158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:17.972308 27142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:10:17.972519 27142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "a62a00f38026409ba35607725cba3a2e"
format_stamp: "Formatted at 2026-08-07 04:10:17 on dist-test-slave-8n49"
server_key: "49fb4f8222c1f6695d193a5e1c981fa2"
server_key_iv: "591aa3c54ead3b49ab53ea58a5967c31"
server_key_version: "encryptionkey@0"
I20260807 04:10:17.972820 27142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:17.986078 27142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:17.990227 27142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:17.990507 27142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:18.014055 27142 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:44645
I20260807 04:10:18.014076 27222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:44645 every 8 connection(s)
I20260807 04:10:18.015517 27142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:18.019470 27223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:18.024876 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 27142
I20260807 04:10:18.025122 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:18.025907 19204 external_mini_cluster.cc:1428] Setting key 63d165a808ebdc437733107436b23588
I20260807 04:10:18.031666 27223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Bootstrap starting.
I20260807 04:10:18.035575 27223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:18.036908 27223 log.cc:824] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:18.040488 27223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: No bootstrap required, opened a new log
Aug 07 04:10:18 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:18.051313 27226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:18.028260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.18.193.62:44645)
0807 04:10:18.029223 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:18.029234 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:18.029280 (+ 46us) server_negotiation.cc:407] Connection header received
0807 04:10:18.031068 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:18.031091 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:18.031490 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:18.031885 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:18.033713 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.035074 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.036926 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.037972 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.038209 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:18.043020 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:18.043031 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:18.043057 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:18.043118 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:18.046758 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:18.047763 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:18.047769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:18.047776 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:18.047948 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:18.048408 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:18.048411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:18.048412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:18.049054 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:18.049297 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:18.049807 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 04:10:18.050223 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":490,"threads_started":1}
I20260807 04:10:18.051280 27223 raft_consensus.cc:348] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62a00f38026409ba35607725cba3a2e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44645 } }
I20260807 04:10:18.052052 27223 raft_consensus.cc:374] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:18.052206 27223 raft_consensus.cc:729] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62a00f38026409ba35607725cba3a2e, State: Initialized, Role: FOLLOWER
I20260807 04:10:18.052918 27223 consensus_queue.cc:260] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62a00f38026409ba35607725cba3a2e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44645 } }
I20260807 04:10:18.053263 27223 raft_consensus.cc:388] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:18.053423 27223 raft_consensus.cc:482] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:18.053630 27223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:18.056352 27223 raft_consensus.cc:504] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62a00f38026409ba35607725cba3a2e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44645 } }
I20260807 04:10:18.056886 27223 leader_election.cc:302] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a62a00f38026409ba35607725cba3a2e; no voters:
I20260807 04:10:18.057508 27223 leader_election.cc:288] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:18.057672 27228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:18.058408 27228 raft_consensus.cc:686] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [term 1 LEADER]: Becoming Leader. State: Replica: a62a00f38026409ba35607725cba3a2e, State: Running, Role: LEADER
I20260807 04:10:18.058945 27228 consensus_queue.cc:237] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62a00f38026409ba35607725cba3a2e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44645 } }
I20260807 04:10:18.059645 27223 sys_catalog.cc:564] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:18.061889 27229 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a62a00f38026409ba35607725cba3a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62a00f38026409ba35607725cba3a2e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44645 } } }
I20260807 04:10:18.061815 27230 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a62a00f38026409ba35607725cba3a2e. Latest consensus state: current_term: 1 leader_uuid: "a62a00f38026409ba35607725cba3a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62a00f38026409ba35607725cba3a2e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44645 } } }
I20260807 04:10:18.062312 27229 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:18.062318 27230 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:18.064106 27236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:18.070580 27236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:18.079416 27236 catalog_manager.cc:1347] Generated new cluster ID: 0d25d8b0e266422698a52648be58f5d7
I20260807 04:10:18.079610 27236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:18.087987 27236 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:18.089480 27236 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:18.095675 27236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Generated new TSK 0
I20260807 04:10:18.096498 27236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab.
Entry for principal oryx/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab.
I20260807 04:10:18.159852 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:44645
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab
--principal=oryx/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:18.511529 27252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:18.511819 27252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:18.511996 27252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:18.519593 27252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:18.519707 27252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:18.519855 27252 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:10:18.519980 27252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:18.530232 27252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab
--principal=oryx/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:44645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:18.531586 27252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:18.533025 27252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:18.547101 27257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:18.547089 27258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:18.547164 27260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:18.548019 27252 server_base.cc:1034] running on GCE node
I20260807 04:10:18.548761 27252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:18.549809 27252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:18.551065 27252 hybrid_clock.cc:648] HybridClock initialized: now 1786075818550995 us; error 52 us; skew 500 ppm
I20260807 04:10:18.551381 27252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:18 dist-test-slave-8n49 krb5kdc[27129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075818, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:18.560103 27252 init.cc:377] Logged in from keytab as oryx/127.18.193.1@KRBTEST.COM (short username oryx)
I20260807 04:10:18.561888 27252 webserver.cc:460] Webserver started at http://127.18.193.1:45761/ using document root <none> and password file <none>
I20260807 04:10:18.562628 27252 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:18.562719 27252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:18.563086 27252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:18.566272 27252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "30ae5e9edacd43fbb1f80939efde3085"
format_stamp: "Formatted at 2026-08-07 04:10:18 on dist-test-slave-8n49"
server_key: "7a0716f443ff852add2dbd50108e5afc"
server_key_iv: "8cd5eb6901970f68e9e3d6babb4b9379"
server_key_version: "encryptionkey@0"
I20260807 04:10:18.567123 27252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "30ae5e9edacd43fbb1f80939efde3085"
format_stamp: "Formatted at 2026-08-07 04:10:18 on dist-test-slave-8n49"
server_key: "7a0716f443ff852add2dbd50108e5afc"
server_key_iv: "8cd5eb6901970f68e9e3d6babb4b9379"
server_key_version: "encryptionkey@0"
I20260807 04:10:18.572492 27252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:10:18.576537 27267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:18.578327 27252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:18.578538 27252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "30ae5e9edacd43fbb1f80939efde3085"
format_stamp: "Formatted at 2026-08-07 04:10:18 on dist-test-slave-8n49"
server_key: "7a0716f443ff852add2dbd50108e5afc"
server_key_iv: "8cd5eb6901970f68e9e3d6babb4b9379"
server_key_version: "encryptionkey@0"
I20260807 04:10:18.578804 27252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:18.605437 27252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:18.610462 27252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:18.610798 27252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:18.613910 27252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:18.614001 27252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:18.614261 27252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:18.614487 27252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 04:10:18 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075818, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:18.639391 27276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:18.614367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:38971 (local address 127.18.193.62:44645)
0807 04:10:18.614873 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:18.614880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:18.616089 (+ 1209us) server_negotiation.cc:407] Connection header received
0807 04:10:18.618225 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:18.618236 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:18.618349 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:18.618497 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:18.620402 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.621868 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.624436 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.625116 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.625262 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:18.630449 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:18.630455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:18.630459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:18.630499 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:18.634584 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:18.636005 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:18.636009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:18.636011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:18.636226 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:18.636895 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:18.636902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:18.636906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:18.637295 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:18.637463 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:18.638864 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0807 04:10:18.639110 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":258,"threads_started":1}
I20260807 04:10:18.641778 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:18.614972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44645 (local address 127.18.193.1:38971)
0807 04:10:18.615899 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:18.615948 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:18.617921 (+ 1973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:18.618724 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:18.618732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:18.619260 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:18.620172 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:18.620204 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.622105 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.622117 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:18.624283 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:18.624296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.625406 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.625419 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:18.625640 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:18.626614 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:18.626670 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:18.630149 (+ 3479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:18.634812 (+ 4663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:18.634828 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:18.634857 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:18.635811 (+ 954us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:18.636387 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:18.636395 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:18.636400 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:18.636771 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:18.637506 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:18.637525 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:18.638251 (+ 726us) client_negotiation.cc:769] Sending connection context
0807 04:10:18.638745 (+ 494us) client_negotiation.cc:240] Negotiation successful
0807 04:10:18.639604 (+ 859us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":387,"threads_started":1}
I20260807 04:10:18.666431 27276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:18.649769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:48859 (local address 127.18.193.62:44645)
0807 04:10:18.649983 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:18.649993 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:18.650976 (+ 983us) server_negotiation.cc:407] Connection header received
0807 04:10:18.651221 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:18.651230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:18.651307 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:18.651428 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:18.652510 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.653487 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.655864 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.656694 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.656850 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:18.659911 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:18.659921 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:18.659927 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:18.659971 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:18.663533 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:18.664334 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:18.664341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:18.664347 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:18.664492 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:18.664915 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:18.664920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:18.664923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:18.665222 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:18.665391 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:18.665804 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 04:10:18.666046 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 04:10:18.666530 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:18.649954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44645 (local address 127.18.193.1:48859)
0807 04:10:18.650839 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:18.650860 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:18.651072 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:18.651457 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:18.651462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:18.652003 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:18.652334 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:18.652345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.653714 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.653720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:18.655680 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:18.655705 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:18.656965 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:18.656975 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:18.657230 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:18.658102 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:18.658136 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:18.659632 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:18.663695 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:18.663707 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:18.663710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:18.664209 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:18.664658 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:18.664665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:18.664676 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:18.664795 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:18.665353 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:18.665360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:18.665548 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 04:10:18.665713 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 04:10:18.666047 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":521,"threads_started":1}
I20260807 04:10:18.674985 27252 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:35229
I20260807 04:10:18.674993 27394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:35229 every 8 connection(s)
I20260807 04:10:18.676419 27252 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:18.677392 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 27252
I20260807 04:10:18.677613 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:10:18.678439 19204 external_mini_cluster.cc:1428] Setting key 502d3cde69d5af00f707977a3aa470d6
I20260807 04:10:18.679100 27395 heartbeater.cc:347] Connected to a master server at 127.18.193.62:44645
I20260807 04:10:18.679364 27395 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:18.680223 27395 heartbeater.cc:511] Master 127.18.193.62:44645 requested a full tablet report, sending...
I20260807 04:10:18.682415 27176 ts_manager.cc:194] Registered new tserver with Master: 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229)
I20260807 04:10:18.684226 27176 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:38971
WARNING: no policy specified for oryx/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab.
Entry for principal oryx/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab.
I20260807 04:10:18.736444 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:44645
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab
--principal=oryx/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:19.095336 27400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:19.095721 27400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:19.095890 27400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:19.104573 27400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:19.104708 27400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:19.104874 27400 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:10:19.105031 27400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:19.117166 27400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab
--principal=oryx/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:44645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:19.118662 27400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:19.120214 27400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:19.137116 27406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:19.137166 27405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:19.137079 27408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:19.138090 27400 server_base.cc:1034] running on GCE node
I20260807 04:10:19.138906 27400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:19.140020 27400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:19.141258 27400 hybrid_clock.cc:648] HybridClock initialized: now 1786075819141171 us; error 59 us; skew 500 ppm
I20260807 04:10:19.141693 27400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:19 dist-test-slave-8n49 krb5kdc[27129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075819, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:19.145277 27400 init.cc:377] Logged in from keytab as oryx/127.18.193.2@KRBTEST.COM (short username oryx)
I20260807 04:10:19.146966 27400 webserver.cc:460] Webserver started at http://127.18.193.2:42889/ using document root <none> and password file <none>
I20260807 04:10:19.147748 27400 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:19.147886 27400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:19.148234 27400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:19.151674 27400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "18fb1718330646dc872da0b55d34e513"
format_stamp: "Formatted at 2026-08-07 04:10:19 on dist-test-slave-8n49"
server_key: "c27a05a26e825859d0fb317ce334e0ff"
server_key_iv: "2aa6893cd83461a3bd597892f1918154"
server_key_version: "encryptionkey@0"
I20260807 04:10:19.152617 27400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "18fb1718330646dc872da0b55d34e513"
format_stamp: "Formatted at 2026-08-07 04:10:19 on dist-test-slave-8n49"
server_key: "c27a05a26e825859d0fb317ce334e0ff"
server_key_iv: "2aa6893cd83461a3bd597892f1918154"
server_key_version: "encryptionkey@0"
I20260807 04:10:19.157989 27400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:10:19.161887 27416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:19.163586 27400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:10:19.163796 27400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "18fb1718330646dc872da0b55d34e513"
format_stamp: "Formatted at 2026-08-07 04:10:19 on dist-test-slave-8n49"
server_key: "c27a05a26e825859d0fb317ce334e0ff"
server_key_iv: "2aa6893cd83461a3bd597892f1918154"
server_key_version: "encryptionkey@0"
I20260807 04:10:19.164084 27400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:19.191335 27400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:19.196338 27400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:19.196718 27400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:19.199743 27400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:19.199869 27400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:19.200053 27400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:19.200153 27400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:10:19 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075819, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:19.224009 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.200135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:38379 (local address 127.18.193.62:44645)
0807 04:10:19.200779 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:19.200786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:19.202034 (+ 1248us) server_negotiation.cc:407] Connection header received
0807 04:10:19.204088 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:19.204094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:19.204193 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:19.204321 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:19.206492 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.207543 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.211466 (+ 3923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.211888 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.212056 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.216723 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:19.216733 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:19.216739 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:19.216788 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:19.220774 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.221516 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.221521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.221525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.221659 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.222126 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.222129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.222132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.222415 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:19.222601 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:19.223362 (+ 761us) server_negotiation.cc:299] Negotiation successful
0807 04:10:19.223604 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":281,"threads_started":1}
I20260807 04:10:19.225737 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.200920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44645 (local address 127.18.193.2:38379)
0807 04:10:19.201836 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:19.201892 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:19.203784 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:19.204619 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:19.204628 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:19.205250 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:19.206206 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.206236 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.207779 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.207790 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.211297 (+ 3507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.211320 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.212193 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.212202 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.212435 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.213314 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:19.213352 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:19.216373 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:19.220927 (+ 4554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.220936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.220958 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.221380 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.221778 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.221783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.221787 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.221994 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.222562 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:19.222570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:19.222975 (+ 405us) client_negotiation.cc:769] Sending connection context
0807 04:10:19.223255 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 04:10:19.223853 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":384,"threads_started":1}
I20260807 04:10:19.251389 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.233783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:60881 (local address 127.18.193.62:44645)
0807 04:10:19.234028 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:19.234036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:19.235106 (+ 1070us) server_negotiation.cc:407] Connection header received
0807 04:10:19.235418 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:19.235427 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:19.235531 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:19.235646 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:19.237353 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.238362 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.241493 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.241911 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.242012 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.245157 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:19.245164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:19.245177 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:19.245226 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:19.248143 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.249310 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.249314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.249316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.249433 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.249833 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.249842 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.249844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.250173 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:19.250347 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:19.250777 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 04:10:19.251018 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:10:19.251590 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.234279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44645 (local address 127.18.193.2:60881)
0807 04:10:19.234974 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:19.234994 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:19.235191 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:19.235729 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:19.235740 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:19.236610 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:19.237159 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.237181 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.238570 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.238579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.241304 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.241333 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.242145 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.242155 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.242423 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.243222 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:19.243258 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:19.244886 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:19.248287 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.248297 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.248304 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.249130 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.249557 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.249563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.249570 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.249713 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.250298 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:19.250303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:19.250529 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 04:10:19.250674 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 04:10:19.251038 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":402,"threads_started":1}
I20260807 04:10:19.264159 27400 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:41849
I20260807 04:10:19.264181 27539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:41849 every 8 connection(s)
I20260807 04:10:19.265658 27400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:19.267930 27541 heartbeater.cc:347] Connected to a master server at 127.18.193.62:44645
I20260807 04:10:19.268181 27541 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:19.268975 27541 heartbeater.cc:511] Master 127.18.193.62:44645 requested a full tablet report, sending...
I20260807 04:10:19.270501 27176 ts_manager.cc:194] Registered new tserver with Master: 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849)
I20260807 04:10:19.271483 27176 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:38379
I20260807 04:10:19.274210 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 27400
I20260807 04:10:19.274488 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:10:19.275023 19204 external_mini_cluster.cc:1428] Setting key e8502f8844a87273fad11b56c91ecad5
WARNING: no policy specified for oryx/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab.
Entry for principal oryx/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab.
I20260807 04:10:19.336808 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:44645
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab
--principal=oryx/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:10:19.687098 27395 heartbeater.cc:503] Master 127.18.193.62:44645 was elected leader, sending a full tablet report...
W20260807 04:10:19.691490 27545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:19.691802 27545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:19.691977 27545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:19.700269 27545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:19.700423 27545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:19.700606 27545 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:10:19.700743 27545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:19.710863 27545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab
--principal=oryx/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.193.62:44645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:19.712282 27545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:19.713734 27545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:19.727603 27554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:19.728139 27552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:19.728163 27551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:19.728538 27545 server_base.cc:1034] running on GCE node
I20260807 04:10:19.729331 27545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:19.730383 27545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:19.731648 27545 hybrid_clock.cc:648] HybridClock initialized: now 1786075819731570 us; error 66 us; skew 500 ppm
I20260807 04:10:19.731977 27545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:19 dist-test-slave-8n49 krb5kdc[27129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075819, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:19.735556 27545 init.cc:377] Logged in from keytab as oryx/127.18.193.3@KRBTEST.COM (short username oryx)
I20260807 04:10:19.737483 27545 webserver.cc:460] Webserver started at http://127.18.193.3:40837/ using document root <none> and password file <none>
I20260807 04:10:19.738255 27545 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:19.738399 27545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:19.738758 27545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:19.742348 27545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "18a7907a0bcf4793ba8c9cd20e87648f"
format_stamp: "Formatted at 2026-08-07 04:10:19 on dist-test-slave-8n49"
server_key: "92ec6c4da525a2f4c4ab204840b54f6c"
server_key_iv: "ca815229bcd2db2cd51cb1a238cff313"
server_key_version: "encryptionkey@0"
I20260807 04:10:19.743291 27545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "18a7907a0bcf4793ba8c9cd20e87648f"
format_stamp: "Formatted at 2026-08-07 04:10:19 on dist-test-slave-8n49"
server_key: "92ec6c4da525a2f4c4ab204840b54f6c"
server_key_iv: "ca815229bcd2db2cd51cb1a238cff313"
server_key_version: "encryptionkey@0"
I20260807 04:10:19.749392 27545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 04:10:19.753914 27561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:19.755486 27545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:10:19.755728 27545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "18a7907a0bcf4793ba8c9cd20e87648f"
format_stamp: "Formatted at 2026-08-07 04:10:19 on dist-test-slave-8n49"
server_key: "92ec6c4da525a2f4c4ab204840b54f6c"
server_key_iv: "ca815229bcd2db2cd51cb1a238cff313"
server_key_version: "encryptionkey@0"
I20260807 04:10:19.756050 27545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:19.771227 27545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:19.777087 27545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:19.777419 27545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:19.780234 27545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:19.780366 27545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:19.780542 27545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:19.780661 27545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 04:10:19 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075819, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:19.809491 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.781118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:34149 (local address 127.18.193.62:44645)
0807 04:10:19.781645 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:19.781654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:19.782915 (+ 1261us) server_negotiation.cc:407] Connection header received
0807 04:10:19.785285 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:19.785292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:19.785398 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:19.785574 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:19.788524 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.790100 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.792818 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.793650 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.793833 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.799450 (+ 5617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:19.799461 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:19.799470 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:19.799526 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:19.804158 (+ 4632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.805739 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.805746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.805753 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.805945 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.806712 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.806719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.806725 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.807169 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:19.807354 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:19.808650 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0807 04:10:19.808944 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":242,"threads_started":1}
I20260807 04:10:19.813221 27573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.781789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44645 (local address 127.18.193.3:34149)
0807 04:10:19.782708 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:19.782782 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:19.784910 (+ 2128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:19.786090 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:19.786109 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:19.787178 (+ 1069us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:19.788289 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.788326 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.790469 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.790479 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.792538 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.792607 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.793903 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.793913 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.794266 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.795153 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:19.795191 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:19.799080 (+ 3889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:19.804374 (+ 5294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.804403 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.804427 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.805565 (+ 1138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.806105 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.806110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.806113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.806487 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.807344 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:19.807359 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:19.808028 (+ 669us) client_negotiation.cc:769] Sending connection context
0807 04:10:19.809880 (+ 1852us) client_negotiation.cc:240] Negotiation successful
0807 04:10:19.810721 (+ 841us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":400,"threads_started":1}
I20260807 04:10:19.824734 27545 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:45489
I20260807 04:10:19.824862 27690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:45489 every 8 connection(s)
I20260807 04:10:19.826793 27545 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:19.827881 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 27545
I20260807 04:10:19.828104 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:10:19.828707 19204 external_mini_cluster.cc:1428] Setting key b8c646678f0f88deee810a626a9f6546
I20260807 04:10:19.829586 27691 heartbeater.cc:347] Connected to a master server at 127.18.193.62:44645
I20260807 04:10:19.829896 27691 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:19.830946 27691 heartbeater.cc:511] Master 127.18.193.62:44645 requested a full tablet report, sending...
I20260807 04:10:19.832587 27176 ts_manager.cc:194] Registered new tserver with Master: 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489)
I20260807 04:10:19.833701 27176 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:34149
I20260807 04:10:19.839046 27686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.821627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44645 (local address 127.18.193.3:57575)
0807 04:10:19.822442 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:19.822468 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:19.822648 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:19.823020 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:19.823028 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:19.823767 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:19.824297 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.824317 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.825678 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.825688 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.827594 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:19.827626 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.829503 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.829513 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:19.829824 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.830887 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:19.830924 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:19.832531 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:19.836481 (+ 3950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.836485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.836488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.836973 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.837377 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:19.837381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:19.837383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:19.837541 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:19.838159 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:19.838163 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:19.838329 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 04:10:19.838473 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 04:10:19.838728 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":415,"threads_started":1}
I20260807 04:10:19.839046 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:19.821642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:57575 (local address 127.18.193.62:44645)
0807 04:10:19.821904 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:19.821914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:19.822596 (+ 682us) server_negotiation.cc:407] Connection header received
0807 04:10:19.822775 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:19.822779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:19.822863 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:19.822951 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:19.824500 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.825504 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.827859 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.828927 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.829080 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:19.832835 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:19.832846 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:19.832853 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:19.832913 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:19.836314 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.837096 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.837100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.837103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.837245 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:19.837682 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:19.837686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:19.837689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:19.838017 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:19.838182 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:19.838554 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 04:10:19.838767 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 04:10:19.844460 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:10:19 dist-test-slave-8n49 krb5kdc[27129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075819, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:10:19 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:19 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:20.274333 27541 heartbeater.cc:503] Master 127.18.193.62:44645 was elected leader, sending a full tablet report...
I20260807 04:10:20.836488 27691 heartbeater.cc:503] Master 127.18.193.62:44645 was elected leader, sending a full tablet report...
W20260807 04:10:21.035627 27699 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.136s user 0.008s sys 0.011s
W20260807 04:10:21.036976 27572 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:19.893211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.18.193.62:44645)
0807 04:10:19.893483 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:19.893490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:19.893778 (+ 288us) server_negotiation.cc:407] Connection header received
0807 04:10:19.893953 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:19.893958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:19.894044 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:19.894140 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:19.895511 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.896476 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.897790 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:19.898231 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:19.898338 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.036641 (+1138303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53208: BlockingRecv error: recv got EOF from 127.0.0.1:53208 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 04:10:21 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075819, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:21.056243 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.041653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.18.193.62:44645)
0807 04:10:21.041932 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.041939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.042135 (+ 196us) server_negotiation.cc:407] Connection header received
0807 04:10:21.042294 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.042298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.042367 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.042457 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:21.043756 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.044510 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.045967 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.046399 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.046507 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.050775 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:21.050780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:21.050783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:21.050823 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:21.053641 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:21.054367 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:21.054371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:21.054373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:21.054525 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:21.054888 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:21.054892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:21.054894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:21.055169 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:21.055391 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.055686 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.055872 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 04:10:21.061259 27176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:10:21.068881 27176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:21.099532 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.082989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.18.193.3:45489)
0807 04:10:21.083571 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.083583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.088307 (+ 4724us) server_negotiation.cc:407] Connection header received
0807 04:10:21.088803 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.088812 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.089079 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.089286 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.092456 (+ 3170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.094104 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.096796 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.098334 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.098469 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.098694 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.099031 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.099218 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":274,"threads_started":1}
I20260807 04:10:21.100374 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.082465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45489 (local address 127.0.0.1:49284)
0807 04:10:21.088166 (+ 5701us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.088204 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.088469 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.089685 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.089697 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.089740 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.092184 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.092202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.094243 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.094248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.096619 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.096644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.098560 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.098566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.098768 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.098825 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.098967 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.099158 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5458,"thread_start_us":5294,"threads_started":1}
I20260807 04:10:21.100986 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.083263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.18.193.1:35229)
0807 04:10:21.091867 (+ 8604us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.091875 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.093140 (+ 1265us) server_negotiation.cc:407] Connection header received
0807 04:10:21.093249 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.093256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.093481 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.093697 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.094467 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.096034 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.098028 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.099301 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.099433 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.099629 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.100466 (+ 837us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.100654 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8423,"thread_start_us":8235,"threads_started":1}
I20260807 04:10:21.101346 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.082654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:35229 (local address 127.0.0.1:55614)
0807 04:10:21.092947 (+ 10293us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.092974 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.093120 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.093701 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.093707 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.093734 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.094241 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.094252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.096181 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.096187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.097866 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.097878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.099771 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.099778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.099968 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.100006 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.100091 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.100206 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9999,"thread_start_us":9833,"threads_started":1}
I20260807 04:10:21.104830 27320 tablet_service.cc:1467] Processing CreateTablet for tablet a9afe9bd52d049c589e2d4942505c5de (DEFAULT_TABLE table=test-table [id=410ae7e9b26d48c2b5951556ba16a1d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:21.104609 27616 tablet_service.cc:1467] Processing CreateTablet for tablet a9afe9bd52d049c589e2d4942505c5de (DEFAULT_TABLE table=test-table [id=410ae7e9b26d48c2b5951556ba16a1d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:21.105389 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.091862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:41849 (local address 127.0.0.1:38922)
0807 04:10:21.093131 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.093147 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.093320 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.093915 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.093920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.093943 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.094574 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.094589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.096460 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.096465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.098722 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.098738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.101573 (+ 2835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.104631 (+ 3058us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.104773 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.104798 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.104913 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.105066 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"thread_start_us":998,"threads_started":1}
I20260807 04:10:21.105425 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.088901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.18.193.2:41849)
0807 04:10:21.092273 (+ 3372us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.092284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.093231 (+ 947us) server_negotiation.cc:407] Connection header received
0807 04:10:21.093497 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.093503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.093700 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.093904 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.094806 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.096312 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.098886 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.101040 (+ 2154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.101197 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.101377 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.104971 (+ 3594us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.105134 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3181,"thread_start_us":3128,"threads_started":1}
I20260807 04:10:21.106330 27320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9afe9bd52d049c589e2d4942505c5de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.106330 27616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9afe9bd52d049c589e2d4942505c5de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.108532 27468 tablet_service.cc:1467] Processing CreateTablet for tablet a9afe9bd52d049c589e2d4942505c5de (DEFAULT_TABLE table=test-table [id=410ae7e9b26d48c2b5951556ba16a1d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:21.109719 27468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9afe9bd52d049c589e2d4942505c5de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.120754 27716 tablet_bootstrap.cc:492] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Bootstrap starting.
I20260807 04:10:21.123643 27717 tablet_bootstrap.cc:492] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Bootstrap starting.
I20260807 04:10:21.125507 27716 tablet_bootstrap.cc:654] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.126886 27716 log.cc:824] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:21.127370 27717 tablet_bootstrap.cc:654] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.127954 27718 tablet_bootstrap.cc:492] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Bootstrap starting.
I20260807 04:10:21.129083 27717 log.cc:824] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:21.131315 27718 tablet_bootstrap.cc:654] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.132055 27716 tablet_bootstrap.cc:492] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: No bootstrap required, opened a new log
I20260807 04:10:21.132350 27716 ts_tablet_manager.cc:1397] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 04:10:21.134343 27717 tablet_bootstrap.cc:492] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: No bootstrap required, opened a new log
I20260807 04:10:21.134651 27717 ts_tablet_manager.cc:1397] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 04:10:21.135803 27718 log.cc:824] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:21.139185 27718 tablet_bootstrap.cc:492] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: No bootstrap required, opened a new log
I20260807 04:10:21.139568 27718 ts_tablet_manager.cc:1397] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 04:10:21.141610 27716 raft_consensus.cc:348] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.142025 27716 raft_consensus.cc:374] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.142158 27716 raft_consensus.cc:729] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb1718330646dc872da0b55d34e513, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.142802 27716 consensus_queue.cc:260] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.144501 27716 ts_tablet_manager.cc:1428] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Time spent starting tablet: real 0.012s user 0.014s sys 0.000s
I20260807 04:10:21.145391 27717 raft_consensus.cc:348] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.145797 27717 raft_consensus.cc:374] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.145903 27717 raft_consensus.cc:729] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a7907a0bcf4793ba8c9cd20e87648f, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.146938 27717 consensus_queue.cc:260] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.149165 27717 ts_tablet_manager.cc:1428] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260807 04:10:21.151001 27718 raft_consensus.cc:348] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.151477 27718 raft_consensus.cc:374] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.151623 27718 raft_consensus.cc:729] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ae5e9edacd43fbb1f80939efde3085, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.152485 27718 consensus_queue.cc:260] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.154189 27718 ts_tablet_manager.cc:1428] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260807 04:10:21.172576 27723 raft_consensus.cc:482] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:21.172904 27723 raft_consensus.cc:504] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.174649 27723 leader_election.cc:288] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229), 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849)
W20260807 04:10:21.180346 27396 tablet.cc:2369] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:10:21.181881 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.175138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:45409 (local address 127.18.193.1:35229)
0807 04:10:21.175619 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.175627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.176044 (+ 417us) server_negotiation.cc:407] Connection header received
0807 04:10:21.176303 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.176309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.176406 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.176485 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.177286 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.178010 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.180008 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.180954 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.181142 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.181219 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.181493 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.181632 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310}
I20260807 04:10:21.182273 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.174888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:35229 (local address 127.18.193.3:45409)
0807 04:10:21.175900 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.175932 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.176102 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.176527 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.176535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.176585 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.177096 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.177117 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.178156 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.178164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.179819 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.179831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.181172 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.181179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.181335 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.181358 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.181465 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.181701 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":124,"thread_start_us":609,"threads_started":1}
I20260807 04:10:21.182354 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.175252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:45003 (local address 127.18.193.2:41849)
0807 04:10:21.175493 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.175504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.176185 (+ 681us) server_negotiation.cc:407] Connection header received
0807 04:10:21.176387 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.176394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.176495 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.176640 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.177303 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.178082 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.179963 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.181182 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.181343 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.181395 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.181982 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.182125 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:10:21.183034 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.175125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:41849 (local address 127.18.193.3:45003)
0807 04:10:21.176064 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.176082 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.176230 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.176665 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.176670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.176731 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.177139 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.177148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.178230 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.178234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.179786 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.179811 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.181557 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.181563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.181772 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.181812 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.182507 (+ 695us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.182644 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":22,"thread_start_us":581,"threads_started":1}
I20260807 04:10:21.183486 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9afe9bd52d049c589e2d4942505c5de" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085" is_pre_election: true
I20260807 04:10:21.184118 27341 raft_consensus.cc:2454] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a7907a0bcf4793ba8c9cd20e87648f in term 0.
I20260807 04:10:21.184140 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9afe9bd52d049c589e2d4942505c5de" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb1718330646dc872da0b55d34e513" is_pre_election: true
I20260807 04:10:21.184763 27493 raft_consensus.cc:2454] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a7907a0bcf4793ba8c9cd20e87648f in term 0.
I20260807 04:10:21.184819 27565 leader_election.cc:302] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a7907a0bcf4793ba8c9cd20e87648f, 30ae5e9edacd43fbb1f80939efde3085; no voters:
I20260807 04:10:21.185305 27723 raft_consensus.cc:2781] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:21.185454 27723 raft_consensus.cc:482] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:21.185585 27723 raft_consensus.cc:3037] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.187840 27723 raft_consensus.cc:504] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.188525 27723 leader_election.cc:288] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 election: Requested vote from peers 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229), 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849)
I20260807 04:10:21.189097 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9afe9bd52d049c589e2d4942505c5de" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085"
I20260807 04:10:21.189126 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9afe9bd52d049c589e2d4942505c5de" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb1718330646dc872da0b55d34e513"
I20260807 04:10:21.189333 27341 raft_consensus.cc:3037] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.189333 27493 raft_consensus.cc:3037] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.192744 27341 raft_consensus.cc:2454] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a7907a0bcf4793ba8c9cd20e87648f in term 1.
I20260807 04:10:21.192744 27493 raft_consensus.cc:2454] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a7907a0bcf4793ba8c9cd20e87648f in term 1.
I20260807 04:10:21.193209 27564 leader_election.cc:302] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a7907a0bcf4793ba8c9cd20e87648f, 18fb1718330646dc872da0b55d34e513; no voters:
I20260807 04:10:21.193552 27723 raft_consensus.cc:2781] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:21.194056 27723 raft_consensus.cc:686] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 LEADER]: Becoming Leader. State: Replica: 18a7907a0bcf4793ba8c9cd20e87648f, State: Running, Role: LEADER
I20260807 04:10:21.194588 27723 consensus_queue.cc:237] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } }
I20260807 04:10:21.199043 27175 catalog_manager.cc:5495] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f reported cstate change: term changed from 0 to 1, leader changed from <none> to 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3). New cstate: current_term: 1 leader_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } health_report { overall_health: UNKNOWN } } }
I20260807 04:10:21.219957 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.214338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.18.193.3:45489)
0807 04:10:21.214556 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.214565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.214659 (+ 94us) server_negotiation.cc:407] Connection header received
0807 04:10:21.214972 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.214976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.215064 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.215228 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:21.215588 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.216345 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.217763 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.218160 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.218317 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.218788 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:21.219341 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:21.219499 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.219583 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.219673 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:10:21.228624 27341 raft_consensus.cc:1261] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Refusing update from remote peer 18a7907a0bcf4793ba8c9cd20e87648f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:21.228632 27493 raft_consensus.cc:1261] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Refusing update from remote peer 18a7907a0bcf4793ba8c9cd20e87648f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:21.229732 27723 consensus_queue.cc:1035] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:21.230075 27727 consensus_queue.cc:1035] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:21.260731 27175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:21.261081 27175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:21.265375 27175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Sending DeleteTablet for 3 replicas of tablet a9afe9bd52d049c589e2d4942505c5de
I20260807 04:10:21.266633 27320 tablet_service.cc:1514] Processing DeleteTablet for tablet a9afe9bd52d049c589e2d4942505c5de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:21 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:55614
I20260807 04:10:21.266639 27616 tablet_service.cc:1514] Processing DeleteTablet for tablet a9afe9bd52d049c589e2d4942505c5de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:21 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:49284
I20260807 04:10:21.267673 27743 tablet_replica.cc:330] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: stopping tablet replica
I20260807 04:10:21.268270 27741 tablet_replica.cc:330] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: stopping tablet replica
I20260807 04:10:21.268256 27468 tablet_service.cc:1514] Processing DeleteTablet for tablet a9afe9bd52d049c589e2d4942505c5de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:21 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38922
I20260807 04:10:21.269127 27743 raft_consensus.cc:2229] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:21.269599 27741 raft_consensus.cc:2229] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:10:21.270483 27741 raft_consensus.cc:2258] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:21.271086 27743 raft_consensus.cc:2258] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:21.273743 27743 ts_tablet_manager.cc:1905] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:21.276571 27175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:10:21.277493 27175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:21.277807 27741 ts_tablet_manager.cc:1905] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:21.284796 27743 ts_tablet_manager.cc:1918] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:21.285108 27743 log.cc:1196] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/a9afe9bd52d049c589e2d4942505c5de
I20260807 04:10:21.285760 27743 ts_tablet_manager.cc:1939] T a9afe9bd52d049c589e2d4942505c5de P 30ae5e9edacd43fbb1f80939efde3085: Deleting consensus metadata
I20260807 04:10:21.285748 27616 tablet_service.cc:1467] Processing CreateTablet for tablet e3b3dfe6ab134f4486b6619d01cd2920 (DEFAULT_TABLE table=test-table [id=b854291deb35442283427404ce9d0c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:21.286280 27616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b3dfe6ab134f4486b6619d01cd2920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.286358 27467 tablet_service.cc:1467] Processing CreateTablet for tablet e3b3dfe6ab134f4486b6619d01cd2920 (DEFAULT_TABLE table=test-table [id=b854291deb35442283427404ce9d0c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:21.286916 27467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b3dfe6ab134f4486b6619d01cd2920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.288096 27161 catalog_manager.cc:4841] TS 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229): tablet a9afe9bd52d049c589e2d4942505c5de (table test-table [id=410ae7e9b26d48c2b5951556ba16a1d3]) successfully deleted
I20260807 04:10:21.288949 27320 tablet_service.cc:1467] Processing CreateTablet for tablet e3b3dfe6ab134f4486b6619d01cd2920 (DEFAULT_TABLE table=test-table [id=b854291deb35442283427404ce9d0c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:21.289376 27320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b3dfe6ab134f4486b6619d01cd2920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.293290 27717 tablet_bootstrap.cc:492] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Bootstrap starting.
I20260807 04:10:21.295071 27717 tablet_bootstrap.cc:654] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.296203 27718 tablet_bootstrap.cc:492] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: Bootstrap starting.
I20260807 04:10:21.297386 27716 tablet_bootstrap.cc:492] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Bootstrap starting.
I20260807 04:10:21.297883 27718 tablet_bootstrap.cc:654] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.299474 27716 tablet_bootstrap.cc:654] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.310943 27718 tablet_bootstrap.cc:492] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: No bootstrap required, opened a new log
I20260807 04:10:21.311723 27718 ts_tablet_manager.cc:1397] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.005s
I20260807 04:10:21.312922 27718 raft_consensus.cc:348] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.313252 27718 raft_consensus.cc:374] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.313373 27718 raft_consensus.cc:729] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ae5e9edacd43fbb1f80939efde3085, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.313701 27718 consensus_queue.cc:260] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.314278 27744 tablet_replica.cc:330] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: stopping tablet replica
I20260807 04:10:21.314429 27718 ts_tablet_manager.cc:1428] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260807 04:10:21.314888 27744 raft_consensus.cc:2229] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:21.317214 27744 raft_consensus.cc:2258] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:21.317414 27716 tablet_bootstrap.cc:492] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: No bootstrap required, opened a new log
I20260807 04:10:21.317674 27716 ts_tablet_manager.cc:1397] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.010s
I20260807 04:10:21.318691 27716 raft_consensus.cc:348] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.319033 27716 raft_consensus.cc:374] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.319193 27716 raft_consensus.cc:729] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb1718330646dc872da0b55d34e513, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.319518 27716 consensus_queue.cc:260] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.319917 27741 ts_tablet_manager.cc:1918] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:21.320134 27741 log.cc:1196] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/a9afe9bd52d049c589e2d4942505c5de
I20260807 04:10:21.320343 27716 ts_tablet_manager.cc:1428] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 04:10:21.322427 27741 ts_tablet_manager.cc:1939] T a9afe9bd52d049c589e2d4942505c5de P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting consensus metadata
I20260807 04:10:21.323041 27744 ts_tablet_manager.cc:1905] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:21.324445 27160 catalog_manager.cc:4841] TS 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489): tablet a9afe9bd52d049c589e2d4942505c5de (table test-table [id=410ae7e9b26d48c2b5951556ba16a1d3]) successfully deleted
I20260807 04:10:21.330050 27717 tablet_bootstrap.cc:492] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: No bootstrap required, opened a new log
I20260807 04:10:21.330257 27717 ts_tablet_manager.cc:1397] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.011s
I20260807 04:10:21.331185 27744 ts_tablet_manager.cc:1918] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:21.331032 27717 raft_consensus.cc:348] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.331374 27744 log.cc:1196] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/a9afe9bd52d049c589e2d4942505c5de
I20260807 04:10:21.331437 27717 raft_consensus.cc:374] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.331595 27717 raft_consensus.cc:729] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a7907a0bcf4793ba8c9cd20e87648f, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.331980 27744 ts_tablet_manager.cc:1939] T a9afe9bd52d049c589e2d4942505c5de P 18fb1718330646dc872da0b55d34e513: Deleting consensus metadata
I20260807 04:10:21.331926 27717 consensus_queue.cc:260] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.332710 27717 ts_tablet_manager.cc:1428] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260807 04:10:21.334069 27692 tablet.cc:2369] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:10:21.335099 27160 catalog_manager.cc:4841] TS 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849): tablet a9afe9bd52d049c589e2d4942505c5de (table test-table [id=410ae7e9b26d48c2b5951556ba16a1d3]) successfully deleted
I20260807 04:10:21.515574 27722 raft_consensus.cc:482] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:21.515822 27722 raft_consensus.cc:504] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.517913 27722 leader_election.cc:288] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229), 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489)
W20260807 04:10:21.520255 27542 tablet.cc:2369] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:10:21.526490 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.518895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:36045 (local address 127.18.193.1:35229)
0807 04:10:21.519124 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.519133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.519160 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:10:21.519267 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.519272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.519349 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.519453 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.520792 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.521962 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.524188 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.525363 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.525524 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.525599 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.526038 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.526199 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 04:10:21.526575 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.518176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:35229 (local address 127.18.193.2:36045)
0807 04:10:21.518832 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.518859 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.519022 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.519491 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.519496 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.519528 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.519945 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.519958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.522163 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.522168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.523995 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.524009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.525652 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.525661 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.525840 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.525876 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.525991 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.526144 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":88,"thread_start_us":330,"threads_started":1}
I20260807 04:10:21.527486 27750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.518176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45489 (local address 127.18.193.2:34807)
0807 04:10:21.519534 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.519552 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.519695 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.520093 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.520101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.520126 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.520595 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.520610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.522141 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.522147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.524614 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.524634 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.526466 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.526475 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.526845 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.526880 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.526963 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.527088 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1235,"mutex_wait_us":279,"thread_start_us":839,"threads_started":1}
I20260807 04:10:21.527486 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.518353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:34807 (local address 127.18.193.3:45489)
0807 04:10:21.518571 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.518579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.519650 (+ 1071us) server_negotiation.cc:407] Connection header received
0807 04:10:21.519857 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.519865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.519957 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.520059 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.520792 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.521992 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.524804 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.526050 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.526222 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.526293 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.527024 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.527186 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:10:21.527877 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b3dfe6ab134f4486b6619d01cd2920" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085" is_pre_election: true
I20260807 04:10:21.528173 27341 raft_consensus.cc:2454] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 0.
I20260807 04:10:21.528754 27419 leader_election.cc:302] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb1718330646dc872da0b55d34e513, 30ae5e9edacd43fbb1f80939efde3085; no voters:
I20260807 04:10:21.528761 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b3dfe6ab134f4486b6619d01cd2920" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" is_pre_election: true
I20260807 04:10:21.529206 27722 raft_consensus.cc:2781] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:21.529336 27722 raft_consensus.cc:482] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:21.529435 27722 raft_consensus.cc:3037] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.529314 27636 raft_consensus.cc:2454] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 0.
I20260807 04:10:21.531778 27722 raft_consensus.cc:504] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.532436 27722 leader_election.cc:288] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 election: Requested vote from peers 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229), 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489)
I20260807 04:10:21.533046 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b3dfe6ab134f4486b6619d01cd2920" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085"
I20260807 04:10:21.533125 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b3dfe6ab134f4486b6619d01cd2920" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a7907a0bcf4793ba8c9cd20e87648f"
I20260807 04:10:21.533284 27341 raft_consensus.cc:3037] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.533355 27636 raft_consensus.cc:3037] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.535779 27341 raft_consensus.cc:2454] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 1.
I20260807 04:10:21.536311 27419 leader_election.cc:302] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb1718330646dc872da0b55d34e513, 30ae5e9edacd43fbb1f80939efde3085; no voters:
I20260807 04:10:21.536774 27636 raft_consensus.cc:2454] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 1.
I20260807 04:10:21.536913 27722 raft_consensus.cc:2781] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:21.544205 27722 raft_consensus.cc:686] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 1 LEADER]: Becoming Leader. State: Replica: 18fb1718330646dc872da0b55d34e513, State: Running, Role: LEADER
I20260807 04:10:21.544970 27722 consensus_queue.cc:237] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.548911 27176 catalog_manager.cc:5495] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18fb1718330646dc872da0b55d34e513 (127.18.193.2). New cstate: current_term: 1 leader_uuid: "18fb1718330646dc872da0b55d34e513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } health_report { overall_health: UNKNOWN } } }
Aug 07 04:10:21 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075817, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.62@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:21.640096 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.622941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44645 (local address 127.0.0.1:53232)
0807 04:10:21.623779 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.623796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.623935 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.624403 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.624407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.625161 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:21.625352 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.625361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.626909 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.626915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.627877 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.627890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.628965 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.628970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.629114 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.629987 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:21.630007 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:21.632682 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:21.637045 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:21.637058 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:21.637065 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:21.637551 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:21.638070 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:21.638078 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:21.638082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:21.638334 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:21.639146 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:21.639156 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:21.639335 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.639490 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.639778 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":541,"threads_started":1}
I20260807 04:10:21.640498 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.623030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.18.193.62:44645)
0807 04:10:21.623557 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.623569 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.623893 (+ 324us) server_negotiation.cc:407] Connection header received
0807 04:10:21.624063 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.624070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.624154 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.624301 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:21.625534 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.626726 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.628070 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.628773 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.628902 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.632913 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:21.632920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:21.632926 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:21.632972 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:21.636854 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:21.637698 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:21.637704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:21.637707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:21.637879 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:21.638487 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:21.638493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:21.638496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:21.638910 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:21.639117 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.639631 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.640003 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":365,"thread_start_us":222,"threads_started":1}
I20260807 04:10:21.646669 27176 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:53232:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 04:10:21.648054 27176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:21.655593 27320 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf59071b9944a6d919480506f98bb81 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=46884a2fbef9489091b5463cfd4ae646]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:10:21.655786 27468 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf59071b9944a6d919480506f98bb81 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=46884a2fbef9489091b5463cfd4ae646]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:10:21.655925 27616 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf59071b9944a6d919480506f98bb81 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=46884a2fbef9489091b5463cfd4ae646]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:10:21.656255 27320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf59071b9944a6d919480506f98bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.656416 27468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf59071b9944a6d919480506f98bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.656462 27616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf59071b9944a6d919480506f98bb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:21.664134 27717 tablet_bootstrap.cc:492] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: Bootstrap starting.
I20260807 04:10:21.664165 27716 tablet_bootstrap.cc:492] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: Bootstrap starting.
I20260807 04:10:21.666006 27716 tablet_bootstrap.cc:654] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.666018 27717 tablet_bootstrap.cc:654] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.667981 27718 tablet_bootstrap.cc:492] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: Bootstrap starting.
I20260807 04:10:21.669870 27718 tablet_bootstrap.cc:654] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:21.674717 27716 tablet_bootstrap.cc:492] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: No bootstrap required, opened a new log
I20260807 04:10:21.675060 27716 ts_tablet_manager.cc:1397] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20260807 04:10:21.675922 27716 raft_consensus.cc:348] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.676292 27716 raft_consensus.cc:374] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.676429 27716 raft_consensus.cc:729] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb1718330646dc872da0b55d34e513, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.676832 27716 consensus_queue.cc:260] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.677398 27722 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:21.677697 27722 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:10:21.679098 27716 ts_tablet_manager.cc:1428] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260807 04:10:21.684139 27718 tablet_bootstrap.cc:492] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: No bootstrap required, opened a new log
I20260807 04:10:21.684325 27718 ts_tablet_manager.cc:1397] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260807 04:10:21.685189 27718 raft_consensus.cc:348] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.685457 27718 raft_consensus.cc:374] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.685549 27718 raft_consensus.cc:729] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ae5e9edacd43fbb1f80939efde3085, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.685889 27718 consensus_queue.cc:260] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.686383 27724 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:21.686771 27724 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:10:21.689254 27717 tablet_bootstrap.cc:492] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: No bootstrap required, opened a new log
I20260807 04:10:21.689476 27717 ts_tablet_manager.cc:1397] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.008s
I20260807 04:10:21.690423 27717 raft_consensus.cc:348] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.690730 27718 ts_tablet_manager.cc:1428] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260807 04:10:21.690769 27717 raft_consensus.cc:374] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:21.690938 27717 raft_consensus.cc:729] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a7907a0bcf4793ba8c9cd20e87648f, State: Initialized, Role: FOLLOWER
I20260807 04:10:21.691268 27717 consensus_queue.cc:260] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.691949 27717 ts_tablet_manager.cc:1428] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:10:21.691836 27723 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:21.692142 27723 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:10:21.940884 27724 raft_consensus.cc:482] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:21.941174 27724 raft_consensus.cc:504] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.942662 27724 leader_election.cc:288] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849), 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489)
I20260807 04:10:21.949330 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.943093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:37653 (local address 127.18.193.2:41849)
0807 04:10:21.943583 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.943588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.943682 (+ 94us) server_negotiation.cc:407] Connection header received
0807 04:10:21.943950 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.943956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.944048 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.944204 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.944790 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.945654 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.947361 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.948150 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.948373 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.948464 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.948820 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.948966 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":270,"threads_started":1}
I20260807 04:10:21.949368 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.943023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:41849 (local address 127.18.193.1:37653)
0807 04:10:21.943545 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.943567 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.943736 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.944223 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.944228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.944254 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.944631 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.944643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.945785 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.945793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.947200 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.947213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.948412 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.948422 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.948641 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.948678 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.948785 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.948949 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":293,"threads_started":1}
I20260807 04:10:21.949915 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.943942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:55573 (local address 127.18.193.3:45489)
0807 04:10:21.944140 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:21.944146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:21.944427 (+ 281us) server_negotiation.cc:407] Connection header received
0807 04:10:21.944670 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:21.944674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:21.944733 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:21.944831 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:21.945473 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.946145 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.947817 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.948843 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.948992 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.949059 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:21.949490 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 04:10:21.949666 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:10:21.950382 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf59071b9944a6d919480506f98bb81" candidate_uuid: "30ae5e9edacd43fbb1f80939efde3085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb1718330646dc872da0b55d34e513" is_pre_election: true
I20260807 04:10:21.950596 27766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:21.943749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45489 (local address 127.18.193.1:55573)
0807 04:10:21.944296 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:21.944331 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:21.944484 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:21.944850 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:21.944854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:21.944877 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:21.945299 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.945309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.946288 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.946294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.947643 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:21.947656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:21.949069 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:21.949078 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:21.949267 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:21.949300 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:10:21.950101 (+ 801us) client_negotiation.cc:240] Negotiation successful
0807 04:10:21.950244 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"spinlock_wait_cycles":1920,"thread_start_us":387,"threads_started":1}
I20260807 04:10:21.950682 27493 raft_consensus.cc:2454] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ae5e9edacd43fbb1f80939efde3085 in term 0.
I20260807 04:10:21.951195 27269 leader_election.cc:302] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb1718330646dc872da0b55d34e513, 30ae5e9edacd43fbb1f80939efde3085; no voters:
I20260807 04:10:21.951226 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf59071b9944a6d919480506f98bb81" candidate_uuid: "30ae5e9edacd43fbb1f80939efde3085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" is_pre_election: true
I20260807 04:10:21.951467 27636 raft_consensus.cc:2454] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ae5e9edacd43fbb1f80939efde3085 in term 0.
I20260807 04:10:21.951611 27724 raft_consensus.cc:2781] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:21.951736 27724 raft_consensus.cc:482] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:21.951867 27724 raft_consensus.cc:3037] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.954425 27724 raft_consensus.cc:504] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.954994 27724 leader_election.cc:288] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [CANDIDATE]: Term 1 election: Requested vote from peers 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849), 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489)
I20260807 04:10:21.955699 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf59071b9944a6d919480506f98bb81" candidate_uuid: "30ae5e9edacd43fbb1f80939efde3085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb1718330646dc872da0b55d34e513"
I20260807 04:10:21.955737 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf59071b9944a6d919480506f98bb81" candidate_uuid: "30ae5e9edacd43fbb1f80939efde3085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a7907a0bcf4793ba8c9cd20e87648f"
I20260807 04:10:21.955920 27493 raft_consensus.cc:3037] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.955963 27636 raft_consensus.cc:3037] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:21.958026 27493 raft_consensus.cc:2454] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ae5e9edacd43fbb1f80939efde3085 in term 1.
I20260807 04:10:21.958058 27636 raft_consensus.cc:2454] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ae5e9edacd43fbb1f80939efde3085 in term 1.
I20260807 04:10:21.958487 27269 leader_election.cc:302] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb1718330646dc872da0b55d34e513, 30ae5e9edacd43fbb1f80939efde3085; no voters:
I20260807 04:10:21.958814 27724 raft_consensus.cc:2781] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:21.959331 27724 raft_consensus.cc:686] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [term 1 LEADER]: Becoming Leader. State: Replica: 30ae5e9edacd43fbb1f80939efde3085, State: Running, Role: LEADER
I20260807 04:10:21.960062 27724 consensus_queue.cc:237] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:21.962422 27767 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: TxnStatusTablet state changed. Reason: New leader 30ae5e9edacd43fbb1f80939efde3085. Latest consensus state: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:21.962860 27767 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:10:21.963562 27769 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 04:10:21.963831 27769 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 04:10:21.963804 27176 catalog_manager.cc:5495] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } health_report { overall_health: UNKNOWN } } }
I20260807 04:10:22.046663 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.039512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:35229 (local address 127.0.0.1:55626)
0807 04:10:22.039800 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:22.039823 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:22.039984 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:22.040740 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:22.040746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:22.040769 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:10:22.040976 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.040987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.042141 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.042152 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.043845 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.043861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.044773 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.044779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.044946 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.045020 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.046040 (+ 1020us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.046065 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:10:22.046188 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 04:10:22.046363 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 04:10:22.046672 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.039638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.18.193.1:35229)
0807 04:10:22.040270 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.040275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.040292 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:10:22.040377 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.040381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.040462 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.040685 (+ 223us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.041136 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.041983 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.044013 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.044609 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.044754 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.045246 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.045924 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.046101 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.046235 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.046331 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":362,"threads_started":1}
I20260807 04:10:22.051460 27493 raft_consensus.cc:1261] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Refusing update from remote peer 30ae5e9edacd43fbb1f80939efde3085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.051795 27636 raft_consensus.cc:1261] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Refusing update from remote peer 30ae5e9edacd43fbb1f80939efde3085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.052318 27767 consensus_queue.cc:1035] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.052778 27724 consensus_queue.cc:1035] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.056656 27723 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: TxnStatusTablet state changed. Reason: New leader 30ae5e9edacd43fbb1f80939efde3085. Latest consensus state: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:22.056967 27723 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:10:22.063243 27773 mvcc.cc:205] Tried to move back new op lower bound from 7315766567115051008 to 7315766566752604160. Current Snapshot: MvccSnapshot[applied={T|T < 7315766567115051008}]
I20260807 04:10:22.066718 27723 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:22.067085 27723 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 18a7907a0bcf4793ba8c9cd20e87648f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:10:22.071430 27767 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:22.071733 27767 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:10:22.075996 27722 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: TxnStatusTablet state changed. Reason: New leader 30ae5e9edacd43fbb1f80939efde3085. Latest consensus state: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:22.076288 27722 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:10:22.076361 27724 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:22.076709 27724 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 30ae5e9edacd43fbb1f80939efde3085: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:10:22.084823 27774 mvcc.cc:205] Tried to move back new op lower bound from 7315766567115051008 to 7315766566752604160. Current Snapshot: MvccSnapshot[applied={T|T < 7315766567115051008}]
I20260807 04:10:22.088619 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.081748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.18.193.2:41849)
0807 04:10:22.081977 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.081986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.082352 (+ 366us) server_negotiation.cc:407] Connection header received
0807 04:10:22.082620 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.082624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.082775 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.083066 (+ 291us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.083577 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.084539 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.086293 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.086795 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.086938 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.087407 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.087964 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.088123 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.088204 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.088289 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:10:22.089247 27722 tablet_replica.cc:439] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30ae5e9edacd43fbb1f80939efde3085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } }
I20260807 04:10:22.089600 27722 tablet_replica.cc:442] T 9cf59071b9944a6d919480506f98bb81 P 18fb1718330646dc872da0b55d34e513: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:10:22.106146 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.099419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:45651 (local address 127.18.193.1:35229)
0807 04:10:22.099631 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.099638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.101231 (+ 1593us) server_negotiation.cc:407] Connection header received
0807 04:10:22.101527 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.101535 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.101649 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.101753 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.102167 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.102930 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.104294 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.104814 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.104942 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.105373 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.105470 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.105570 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.105712 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.105791 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:10:22.106210 27786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.099235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:35229 (local address 127.18.193.2:45651)
0807 04:10:22.101106 (+ 1871us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:22.101122 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:22.101323 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:22.101772 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:22.101777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:22.101787 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:10:22.102025 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.102034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.103072 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.103080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.104155 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.104166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.104980 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.104986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.105121 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.105173 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.105572 (+ 399us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.105586 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 04:10:22.105686 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 04:10:22.105820 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1713,"thread_start_us":1506,"threads_started":1}
I20260807 04:10:22.115891 27636 raft_consensus.cc:1261] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Refusing update from remote peer 18fb1718330646dc872da0b55d34e513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.116062 27341 raft_consensus.cc:1261] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Refusing update from remote peer 18fb1718330646dc872da0b55d34e513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.116642 27722 consensus_queue.cc:1035] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.117221 27722 consensus_queue.cc:1035] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.128171 27775 mvcc.cc:205] Tried to move back new op lower bound from 7315766567380738048 to 7315766565051830272. Current Snapshot: MvccSnapshot[applied={T|T < 7315766567380738048}]
I20260807 04:10:22.162732 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.154855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:35359 (local address 127.18.193.2:41849)
0807 04:10:22.155043 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.155048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.155833 (+ 785us) server_negotiation.cc:407] Connection header received
0807 04:10:22.156064 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.156072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.156159 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.156271 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.157041 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.158224 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.160460 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.161120 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.161246 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.161967 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.162096 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.162197 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.162314 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.162401 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:10:22.162868 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.154897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:41849 (local address 127.18.193.1:35359)
0807 04:10:22.155738 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:22.155760 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:22.155916 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:22.156297 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:22.156304 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:22.156324 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:10:22.156813 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.156827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.158423 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.158440 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.160274 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.160290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.161506 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.161514 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.161692 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.161777 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.162200 (+ 423us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.162214 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 04:10:22.162294 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 04:10:22.162451 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":453,"threads_started":1}
I20260807 04:10:22.200862 27176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:22.201167 27176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:22.204870 27176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Sending DeleteTablet for 3 replicas of tablet e3b3dfe6ab134f4486b6619d01cd2920
I20260807 04:10:22.205727 27320 tablet_service.cc:1514] Processing DeleteTablet for tablet e3b3dfe6ab134f4486b6619d01cd2920 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:55614
I20260807 04:10:22.206255 27468 tablet_service.cc:1514] Processing DeleteTablet for tablet e3b3dfe6ab134f4486b6619d01cd2920 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38922
I20260807 04:10:22.206661 27616 tablet_service.cc:1514] Processing DeleteTablet for tablet e3b3dfe6ab134f4486b6619d01cd2920 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:49284
I20260807 04:10:22.214460 27796 tablet_replica.cc:330] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: stopping tablet replica
I20260807 04:10:22.215415 27796 raft_consensus.cc:2229] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:10:22.216754 27796 raft_consensus.cc:2258] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.220624 27797 tablet_replica.cc:330] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: stopping tablet replica
I20260807 04:10:22.221107 27797 raft_consensus.cc:2229] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:22.221355 27797 raft_consensus.cc:2258] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.222568 27797 ts_tablet_manager.cc:1905] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.228194 27795 tablet_replica.cc:330] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: stopping tablet replica
I20260807 04:10:22.228418 27795 raft_consensus.cc:2229] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:22.228744 27795 raft_consensus.cc:2258] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.230197 27795 ts_tablet_manager.cc:1905] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.231282 27797 ts_tablet_manager.cc:1918] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:10:22.231417 27797 log.cc:1196] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/e3b3dfe6ab134f4486b6619d01cd2920
I20260807 04:10:22.231868 27797 ts_tablet_manager.cc:1939] T e3b3dfe6ab134f4486b6619d01cd2920 P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting consensus metadata
I20260807 04:10:22.233042 27160 catalog_manager.cc:4841] TS 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489): tablet e3b3dfe6ab134f4486b6619d01cd2920 (table test-table [id=b854291deb35442283427404ce9d0c4a]) successfully deleted
I20260807 04:10:22.233367 27796 ts_tablet_manager.cc:1905] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.238193 27795 ts_tablet_manager.cc:1918] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:10:22.238354 27795 log.cc:1196] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/e3b3dfe6ab134f4486b6619d01cd2920
I20260807 04:10:22.238839 27795 ts_tablet_manager.cc:1939] T e3b3dfe6ab134f4486b6619d01cd2920 P 30ae5e9edacd43fbb1f80939efde3085: Deleting consensus metadata
I20260807 04:10:22.240043 27161 catalog_manager.cc:4841] TS 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229): tablet e3b3dfe6ab134f4486b6619d01cd2920 (table test-table [id=b854291deb35442283427404ce9d0c4a]) successfully deleted
I20260807 04:10:22.240598 27796 ts_tablet_manager.cc:1918] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:10:22.240757 27796 log.cc:1196] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/e3b3dfe6ab134f4486b6619d01cd2920
I20260807 04:10:22.241336 27796 ts_tablet_manager.cc:1939] T e3b3dfe6ab134f4486b6619d01cd2920 P 18fb1718330646dc872da0b55d34e513: Deleting consensus metadata
I20260807 04:10:22.242622 27160 catalog_manager.cc:4841] TS 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849): tablet e3b3dfe6ab134f4486b6619d01cd2920 (table test-table [id=b854291deb35442283427404ce9d0c4a]) successfully deleted
Aug 07 04:10:22 dist-test-slave-8n49 krb5kdc[27129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:10:22 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:22 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:10:22.342635 27804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:22.321535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.18.193.62:44645)
0807 04:10:22.329691 (+ 8156us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.329698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.329720 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:10:22.329786 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.329792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.329868 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.329976 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:22.331927 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.333085 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.334453 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.334888 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.335045 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.342283 (+ 7238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53242: BlockingRecv error: recv got EOF from 127.0.0.1:53242 (error 108)
Metrics: {"server-negotiator.queue_time_us":7985,"thread_start_us":8020,"threads_started":1}
Aug 07 04:10:22 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM
I20260807 04:10:22.374305 27804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.351557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.18.193.62:44645)
0807 04:10:22.351780 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.351789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.357297 (+ 5508us) server_negotiation.cc:407] Connection header received
0807 04:10:22.357536 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.357543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.357624 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.357738 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:22.359182 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.360354 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.362076 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.362506 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.362663 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.367198 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:22.367203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:22.367206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:22.367246 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:22.370482 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:22.371197 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:22.371203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:22.371207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:22.371356 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:22.371769 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:22.371774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:22.371778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:22.372336 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:22.372679 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.373868 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.374056 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:10:22.378242 27176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:10:22.378926 27176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:22.386539 27616 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc5298bfb034ba2b6aea077671e716f (DEFAULT_TABLE table=test-table [id=d66a7a0ec5a944b784e0f52021a69019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:22.387048 27616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc5298bfb034ba2b6aea077671e716f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:22.386972 27320 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc5298bfb034ba2b6aea077671e716f (DEFAULT_TABLE table=test-table [id=d66a7a0ec5a944b784e0f52021a69019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:22.387393 27320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc5298bfb034ba2b6aea077671e716f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:22.394927 27468 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc5298bfb034ba2b6aea077671e716f (DEFAULT_TABLE table=test-table [id=d66a7a0ec5a944b784e0f52021a69019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:22.395483 27468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc5298bfb034ba2b6aea077671e716f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:22.398707 27812 tablet_bootstrap.cc:492] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Bootstrap starting.
I20260807 04:10:22.404846 27812 tablet_bootstrap.cc:654] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:22.406798 27813 tablet_bootstrap.cc:492] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: Bootstrap starting.
I20260807 04:10:22.409341 27813 tablet_bootstrap.cc:654] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:22.411513 27811 tablet_bootstrap.cc:492] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: Bootstrap starting.
I20260807 04:10:22.413444 27811 tablet_bootstrap.cc:654] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:22.413849 27812 tablet_bootstrap.cc:492] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: No bootstrap required, opened a new log
I20260807 04:10:22.414039 27812 ts_tablet_manager.cc:1397] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260807 04:10:22.415017 27812 raft_consensus.cc:348] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.415308 27812 raft_consensus.cc:374] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:22.415473 27812 raft_consensus.cc:729] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ae5e9edacd43fbb1f80939efde3085, State: Initialized, Role: FOLLOWER
I20260807 04:10:22.416002 27813 tablet_bootstrap.cc:492] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: No bootstrap required, opened a new log
I20260807 04:10:22.415820 27812 consensus_queue.cc:260] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.416203 27813 ts_tablet_manager.cc:1397] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 04:10:22.417160 27813 raft_consensus.cc:348] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.417485 27813 raft_consensus.cc:374] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:22.417609 27813 raft_consensus.cc:729] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb1718330646dc872da0b55d34e513, State: Initialized, Role: FOLLOWER
I20260807 04:10:22.417886 27813 consensus_queue.cc:260] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.420004 27812 ts_tablet_manager.cc:1428] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260807 04:10:22.420776 27813 ts_tablet_manager.cc:1428] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 04:10:22.423782 27811 tablet_bootstrap.cc:492] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: No bootstrap required, opened a new log
I20260807 04:10:22.423970 27811 ts_tablet_manager.cc:1397] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260807 04:10:22.424749 27811 raft_consensus.cc:348] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.424974 27811 raft_consensus.cc:374] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:22.425074 27811 raft_consensus.cc:729] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a7907a0bcf4793ba8c9cd20e87648f, State: Initialized, Role: FOLLOWER
I20260807 04:10:22.425334 27811 consensus_queue.cc:260] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.426857 27811 ts_tablet_manager.cc:1428] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 04:10:22.497817 27722 raft_consensus.cc:482] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:22.498077 27722 raft_consensus.cc:504] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.498896 27722 leader_election.cc:288] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229), 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489)
I20260807 04:10:22.499538 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc5298bfb034ba2b6aea077671e716f" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085" is_pre_election: true
I20260807 04:10:22.499625 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc5298bfb034ba2b6aea077671e716f" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" is_pre_election: true
I20260807 04:10:22.499820 27341 raft_consensus.cc:2454] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 0.
I20260807 04:10:22.499861 27636 raft_consensus.cc:2454] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 0.
I20260807 04:10:22.500378 27419 leader_election.cc:302] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb1718330646dc872da0b55d34e513, 30ae5e9edacd43fbb1f80939efde3085; no voters:
I20260807 04:10:22.500861 27722 raft_consensus.cc:2781] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:22.500975 27722 raft_consensus.cc:482] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:22.501085 27722 raft_consensus.cc:3037] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:22.504145 27722 raft_consensus.cc:504] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.505337 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc5298bfb034ba2b6aea077671e716f" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a7907a0bcf4793ba8c9cd20e87648f"
I20260807 04:10:22.505333 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc5298bfb034ba2b6aea077671e716f" candidate_uuid: "18fb1718330646dc872da0b55d34e513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085"
I20260807 04:10:22.505553 27636 raft_consensus.cc:3037] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:22.505570 27341 raft_consensus.cc:3037] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:22.508505 27636 raft_consensus.cc:2454] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 1.
I20260807 04:10:22.508642 27341 raft_consensus.cc:2454] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb1718330646dc872da0b55d34e513 in term 1.
I20260807 04:10:22.509032 27722 leader_election.cc:288] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 election: Requested vote from peers 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229), 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489)
I20260807 04:10:22.509080 27418 leader_election.cc:302] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a7907a0bcf4793ba8c9cd20e87648f, 18fb1718330646dc872da0b55d34e513; no voters:
I20260807 04:10:22.509440 27722 raft_consensus.cc:2781] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:22.509677 27722 raft_consensus.cc:686] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 1 LEADER]: Becoming Leader. State: Replica: 18fb1718330646dc872da0b55d34e513, State: Running, Role: LEADER
I20260807 04:10:22.509956 27722 consensus_queue.cc:237] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } }
I20260807 04:10:22.512773 27176 catalog_manager.cc:5495] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18fb1718330646dc872da0b55d34e513 (127.18.193.2). New cstate: current_term: 1 leader_uuid: "18fb1718330646dc872da0b55d34e513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } health_report { overall_health: UNKNOWN } } }
I20260807 04:10:22.531280 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.525124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.18.193.2:41849)
0807 04:10:22.525329 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.525335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.525449 (+ 114us) server_negotiation.cc:407] Connection header received
0807 04:10:22.525662 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.525669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.525765 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.525869 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.526192 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.527343 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.529086 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.529830 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.529966 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.530456 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.530596 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.530686 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.530871 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.530963 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 04:10:22.534161 27636 raft_consensus.cc:1261] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Refusing update from remote peer 18fb1718330646dc872da0b55d34e513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.534224 27341 raft_consensus.cc:1261] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Refusing update from remote peer 18fb1718330646dc872da0b55d34e513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.534946 27790 consensus_queue.cc:1035] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.535336 27722 consensus_queue.cc:1035] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.546995 27175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:22.547307 27175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:22.551543 27175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Sending DeleteTablet for 3 replicas of tablet 1dc5298bfb034ba2b6aea077671e716f
I20260807 04:10:22.552726 27320 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc5298bfb034ba2b6aea077671e716f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:55614
I20260807 04:10:22.553025 27795 tablet_replica.cc:330] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: stopping tablet replica
I20260807 04:10:22.553231 27795 raft_consensus.cc:2229] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:22.553403 27795 pending_rounds.cc:62] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Trying to abort 2 pending ops.
I20260807 04:10:22.553593 27468 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc5298bfb034ba2b6aea077671e716f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38922
I20260807 04:10:22.553545 27795 pending_rounds.cc:68] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315766569094172672 op_type: WRITE_OP write_request { tablet_id: "1dc5298bfb034ba2b6aea077671e716f" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\332\267\325\323\006\"8\n\ntest-admin\022*\n d66a7a0ec5a944b784e0f52021a69019\020\001\030\001 \001(\001" signature: "<redacted>""?\310\r\250H5\305\016j\341\253\311\035\262\014|\210\335\205\202\211<\270\236\267\236\322\212\243\332\236\246P\271\355O\240\006\005\331v3\337\202\255\370\302\204Z\020\206\214\007\325\023\316\336}\352(\302\340\344;" signing_key_seq_num: 0 } } request_id { client_id: "c8815a65acda49b48ca932d8ff79e9ab" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260807 04:10:22.553931 27616 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc5298bfb034ba2b6aea077671e716f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:49284
I20260807 04:10:22.554234 27796 tablet_replica.cc:330] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: stopping tablet replica
I20260807 04:10:22.554247 27797 tablet_replica.cc:330] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: stopping tablet replica
I20260807 04:10:22.554344 27795 pending_rounds.cc:68] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315766569003311104 op_type: NO_OP noop_request { }
I20260807 04:10:22.554457 27796 raft_consensus.cc:2229] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:10:22.554455 27797 raft_consensus.cc:2229] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:22.554500 27795 raft_consensus.cc:2866] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260807 04:10:22.554589 27419 consensus_peers.cc:487] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 -> Peer 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229): Couldn't send request to peer 30ae5e9edacd43fbb1f80939efde3085. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260807 04:10:22.554643 27795 raft_consensus.cc:2258] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.554701 27797 raft_consensus.cc:2258] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.554980 27176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:10:22.555635 27176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:22.557220 27320 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc5298bfb034ba2b6aea077671e716f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:55614
I20260807 04:10:22.557440 27797 ts_tablet_manager.cc:1905] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.562848 27797 ts_tablet_manager.cc:1918] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:22.563040 27797 log.cc:1196] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/1dc5298bfb034ba2b6aea077671e716f
I20260807 04:10:22.563869 27796 raft_consensus.cc:2258] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.564814 27797 ts_tablet_manager.cc:1939] T 1dc5298bfb034ba2b6aea077671e716f P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting consensus metadata
I20260807 04:10:22.565136 27796 ts_tablet_manager.cc:1905] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.565959 27160 catalog_manager.cc:4841] TS 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489): tablet 1dc5298bfb034ba2b6aea077671e716f (table test-table [id=d66a7a0ec5a944b784e0f52021a69019]) successfully deleted
I20260807 04:10:22.567173 27795 ts_tablet_manager.cc:1905] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.568105 27616 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebb361f152646a782a9790063c8479f (DEFAULT_TABLE table=test-table [id=a565cfbbcc6e4f398d042cfe2acad0ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:22.568531 27616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebb361f152646a782a9790063c8479f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:22.569794 27320 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebb361f152646a782a9790063c8479f (DEFAULT_TABLE table=test-table [id=a565cfbbcc6e4f398d042cfe2acad0ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:22.570235 27320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebb361f152646a782a9790063c8479f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:22.572940 27468 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebb361f152646a782a9790063c8479f (DEFAULT_TABLE table=test-table [id=a565cfbbcc6e4f398d042cfe2acad0ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:22.573379 27468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebb361f152646a782a9790063c8479f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:22.574242 27796 ts_tablet_manager.cc:1918] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:22.574469 27796 log.cc:1196] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/1dc5298bfb034ba2b6aea077671e716f
I20260807 04:10:22.575032 27796 ts_tablet_manager.cc:1939] T 1dc5298bfb034ba2b6aea077671e716f P 18fb1718330646dc872da0b55d34e513: Deleting consensus metadata
I20260807 04:10:22.579315 27813 tablet_bootstrap.cc:492] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: Bootstrap starting.
I20260807 04:10:22.579739 27160 catalog_manager.cc:4841] TS 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849): tablet 1dc5298bfb034ba2b6aea077671e716f (table test-table [id=d66a7a0ec5a944b784e0f52021a69019]) successfully deleted
I20260807 04:10:22.581204 27813 tablet_bootstrap.cc:654] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:22.583007 27811 tablet_bootstrap.cc:492] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: Bootstrap starting.
I20260807 04:10:22.584482 27811 tablet_bootstrap.cc:654] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:22.584694 27812 tablet_bootstrap.cc:492] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: Bootstrap starting.
I20260807 04:10:22.585078 27795 ts_tablet_manager.cc:1918] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:22.585227 27795 log.cc:1196] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/1dc5298bfb034ba2b6aea077671e716f
I20260807 04:10:22.585704 27795 ts_tablet_manager.cc:1939] T 1dc5298bfb034ba2b6aea077671e716f P 30ae5e9edacd43fbb1f80939efde3085: Deleting consensus metadata
I20260807 04:10:22.586578 27812 tablet_bootstrap.cc:654] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:22.586925 27161 catalog_manager.cc:4841] TS 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229): tablet 1dc5298bfb034ba2b6aea077671e716f (table test-table [id=d66a7a0ec5a944b784e0f52021a69019]) successfully deleted
W20260807 04:10:22.587903 27161 catalog_manager.cc:4808] TS 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229): delete failed for tablet 1dc5298bfb034ba2b6aea077671e716f because the tablet was not found. No further retry: Not found: Tablet not found: 1dc5298bfb034ba2b6aea077671e716f
I20260807 04:10:22.589996 27813 tablet_bootstrap.cc:492] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: No bootstrap required, opened a new log
I20260807 04:10:22.590322 27813 ts_tablet_manager.cc:1397] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.003s
I20260807 04:10:22.590364 27811 tablet_bootstrap.cc:492] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: No bootstrap required, opened a new log
I20260807 04:10:22.590543 27811 ts_tablet_manager.cc:1397] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 04:10:22.591549 27811 raft_consensus.cc:348] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.591868 27811 raft_consensus.cc:374] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:22.591958 27811 raft_consensus.cc:729] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a7907a0bcf4793ba8c9cd20e87648f, State: Initialized, Role: FOLLOWER
I20260807 04:10:22.591591 27813 raft_consensus.cc:348] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.592180 27813 raft_consensus.cc:374] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:22.592338 27813 raft_consensus.cc:729] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb1718330646dc872da0b55d34e513, State: Initialized, Role: FOLLOWER
I20260807 04:10:22.592244 27811 consensus_queue.cc:260] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.592842 27811 ts_tablet_manager.cc:1428] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 04:10:22.595734 27813 consensus_queue.cc:260] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.597888 27723 raft_consensus.cc:482] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:22.598035 27723 raft_consensus.cc:504] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.598701 27723 leader_election.cc:288] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849), 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229)
I20260807 04:10:22.599445 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebb361f152646a782a9790063c8479f" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb1718330646dc872da0b55d34e513" is_pre_election: true
I20260807 04:10:22.599794 27493 raft_consensus.cc:2454] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a7907a0bcf4793ba8c9cd20e87648f in term 0.
I20260807 04:10:22.600728 27564 leader_election.cc:302] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a7907a0bcf4793ba8c9cd20e87648f, 18fb1718330646dc872da0b55d34e513; no voters:
I20260807 04:10:22.601090 27723 raft_consensus.cc:2781] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:22.601230 27723 raft_consensus.cc:482] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:22.601338 27723 raft_consensus.cc:3037] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:22.601712 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebb361f152646a782a9790063c8479f" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085" is_pre_election: true
I20260807 04:10:22.602397 27813 ts_tablet_manager.cc:1428] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: Time spent starting tablet: real 0.012s user 0.002s sys 0.001s
W20260807 04:10:22.603299 27565 leader_election.cc:341] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229): Illegal state: must be running to vote when last-logged opid is not known
I20260807 04:10:22.604875 27723 raft_consensus.cc:504] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.605957 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebb361f152646a782a9790063c8479f" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb1718330646dc872da0b55d34e513"
I20260807 04:10:22.606176 27493 raft_consensus.cc:3037] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:22.608434 27812 tablet_bootstrap.cc:492] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: No bootstrap required, opened a new log
I20260807 04:10:22.608680 27812 ts_tablet_manager.cc:1397] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.009s
I20260807 04:10:22.609444 27493 raft_consensus.cc:2454] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a7907a0bcf4793ba8c9cd20e87648f in term 1.
I20260807 04:10:22.609486 27812 raft_consensus.cc:348] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.609778 27812 raft_consensus.cc:374] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:22.609881 27812 raft_consensus.cc:729] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ae5e9edacd43fbb1f80939efde3085, State: Initialized, Role: FOLLOWER
I20260807 04:10:22.609925 27564 leader_election.cc:302] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a7907a0bcf4793ba8c9cd20e87648f, 18fb1718330646dc872da0b55d34e513; no voters:
I20260807 04:10:22.610164 27812 consensus_queue.cc:260] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.610664 27824 raft_consensus.cc:2781] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:22.611217 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebb361f152646a782a9790063c8479f" candidate_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae5e9edacd43fbb1f80939efde3085"
I20260807 04:10:22.611440 27341 raft_consensus.cc:3037] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:22.613950 27812 ts_tablet_manager.cc:1428] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 04:10:22.614447 27341 raft_consensus.cc:2454] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a7907a0bcf4793ba8c9cd20e87648f in term 1.
I20260807 04:10:22.614699 27824 raft_consensus.cc:686] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 LEADER]: Becoming Leader. State: Replica: 18a7907a0bcf4793ba8c9cd20e87648f, State: Running, Role: LEADER
I20260807 04:10:22.615099 27824 consensus_queue.cc:237] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } }
I20260807 04:10:22.615423 27723 leader_election.cc:288] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [CANDIDATE]: Term 1 election: Requested vote from peers 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849), 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229)
I20260807 04:10:22.622684 27174 catalog_manager.cc:5495] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f reported cstate change: term changed from 0 to 1, leader changed from <none> to 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3). New cstate: current_term: 1 leader_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18a7907a0bcf4793ba8c9cd20e87648f" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 } health_report { overall_health: UNKNOWN } } }
I20260807 04:10:22.647147 27827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.641635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.18.193.3:45489)
0807 04:10:22.642204 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.642210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.642233 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:10:22.642300 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.642305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.642392 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.642499 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.642901 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.643668 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.645238 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.645799 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.645935 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.646361 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.646472 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.646582 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.646736 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.646811 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":296,"threads_started":1}
I20260807 04:10:22.663913 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.657530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:44925 (local address 127.18.193.1:35229)
0807 04:10:22.658084 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.658092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.658118 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:10:22.658306 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.658312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.658418 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.658558 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.659056 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.660171 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.661827 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.662485 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.662635 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.663046 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.663193 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.663299 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.663490 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.663591 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":332,"threads_started":1}
I20260807 04:10:22.663988 27830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.657276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:35229 (local address 127.18.193.3:44925)
0807 04:10:22.658002 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:22.658022 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:22.658154 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:22.658607 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:22.658611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:22.658622 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:10:22.658889 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.658897 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.660319 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.660323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.661666 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.661680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.662671 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.662675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.662797 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.662855 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.663327 (+ 472us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.663339 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 04:10:22.663430 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 04:10:22.663643 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":307,"threads_started":1}
I20260807 04:10:22.673516 27341 raft_consensus.cc:1261] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Refusing update from remote peer 18a7907a0bcf4793ba8c9cd20e87648f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.673585 27493 raft_consensus.cc:1261] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Refusing update from remote peer 18a7907a0bcf4793ba8c9cd20e87648f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:22.674120 27824 consensus_queue.cc:1035] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ae5e9edacd43fbb1f80939efde3085" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.674439 27825 consensus_queue.cc:1035] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fb1718330646dc872da0b55d34e513" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:22.706804 27827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.700063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:34043 (local address 127.18.193.3:45489)
0807 04:10:22.700294 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.700303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.701167 (+ 864us) server_negotiation.cc:407] Connection header received
0807 04:10:22.701420 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.701425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.701510 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.701574 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:22.702386 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.703113 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.704750 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.705351 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.705450 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.706070 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.706189 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.706240 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.706426 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.706510 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:10:22.706919 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.699778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45489 (local address 127.18.193.1:34043)
0807 04:10:22.701064 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:22.701080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:22.701237 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:22.701785 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:22.701790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:22.701823 (+ 33us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:10:22.702092 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.702107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.703263 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.703266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.704585 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:22.704600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.705611 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.705617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:22.705827 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.705879 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:10:22.706306 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:10:22.706313 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:10:22.706403 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 04:10:22.706536 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"thread_start_us":977,"threads_started":1}
I20260807 04:10:22.754303 27174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:22.754627 27174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:22.758126 27174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a62a00f38026409ba35607725cba3a2e: Sending DeleteTablet for 3 replicas of tablet 4ebb361f152646a782a9790063c8479f
I20260807 04:10:22.759083 27468 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ebb361f152646a782a9790063c8479f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:38922
I20260807 04:10:22.759270 27616 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ebb361f152646a782a9790063c8479f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:49284
I20260807 04:10:22.759480 27796 tablet_replica.cc:330] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: stopping tablet replica
I20260807 04:10:22.759624 27797 tablet_replica.cc:330] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: stopping tablet replica
I20260807 04:10:22.759694 27796 raft_consensus.cc:2229] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:22.759851 27797 raft_consensus.cc:2229] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:10:22.759944 27796 raft_consensus.cc:2258] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.760221 27797 raft_consensus.cc:2258] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.760344 27320 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ebb361f152646a782a9790063c8479f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:22 UTC) from {username='oryx', principal='oryx/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:55614
I20260807 04:10:22.760807 27795 tablet_replica.cc:330] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: stopping tablet replica
I20260807 04:10:22.761021 27795 raft_consensus.cc:2229] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:22.761260 27795 raft_consensus.cc:2258] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:22.761416 27796 ts_tablet_manager.cc:1905] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.762143 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 27252
I20260807 04:10:22.762494 27797 ts_tablet_manager.cc:1905] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.770094 27797 ts_tablet_manager.cc:1918] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:10:22.770257 27797 log.cc:1196] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/4ebb361f152646a782a9790063c8479f
I20260807 04:10:22.770730 27797 ts_tablet_manager.cc:1939] T 4ebb361f152646a782a9790063c8479f P 18a7907a0bcf4793ba8c9cd20e87648f: Deleting consensus metadata
I20260807 04:10:22.770757 27796 ts_tablet_manager.cc:1918] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:10:22.770912 27796 log.cc:1196] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/4ebb361f152646a782a9790063c8479f
I20260807 04:10:22.771505 27796 ts_tablet_manager.cc:1939] T 4ebb361f152646a782a9790063c8479f P 18fb1718330646dc872da0b55d34e513: Deleting consensus metadata
I20260807 04:10:22.771991 27160 catalog_manager.cc:4841] TS 18a7907a0bcf4793ba8c9cd20e87648f (127.18.193.3:45489): tablet 4ebb361f152646a782a9790063c8479f (table test-table [id=a565cfbbcc6e4f398d042cfe2acad0ec]) successfully deleted
I20260807 04:10:22.772397 27795 ts_tablet_manager.cc:1905] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:22.772802 27160 catalog_manager.cc:4841] TS 18fb1718330646dc872da0b55d34e513 (127.18.193.2:41849): tablet 4ebb361f152646a782a9790063c8479f (table test-table [id=a565cfbbcc6e4f398d042cfe2acad0ec]) successfully deleted
I20260807 04:10:22.779253 27795 ts_tablet_manager.cc:1918] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:10:22.779534 27795 log.cc:1196] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/4ebb361f152646a782a9790063c8479f
I20260807 04:10:22.780032 27795 ts_tablet_manager.cc:1939] T 4ebb361f152646a782a9790063c8479f P 30ae5e9edacd43fbb1f80939efde3085: Deleting consensus metadata
I20260807 04:10:22.781335 27161 catalog_manager.cc:4841] TS 30ae5e9edacd43fbb1f80939efde3085 (127.18.193.1:35229): tablet 4ebb361f152646a782a9790063c8479f (table test-table [id=a565cfbbcc6e4f398d042cfe2acad0ec]) successfully deleted
Aug 07 04:10:22 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.1@KRBTEST.COM
I20260807 04:10:22.793502 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.763791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.18.193.1:35229)
0807 04:10:22.763980 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.763987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.773134 (+ 9147us) server_negotiation.cc:407] Connection header received
0807 04:10:22.773354 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.773359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.773457 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.773590 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:22.775207 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.776441 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.778558 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.779210 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.779355 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.785127 (+ 5772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:22.785149 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:22.785177 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:22.785230 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:22.789997 (+ 4767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:22.790891 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:22.790901 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:22.790905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:22.791096 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:22.791601 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:22.791606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:22.791610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:22.792138 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:22.792459 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.792905 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.793143 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 04:10:22.794250 27389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:22.925786 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 27252
I20260807 04:10:22.951480 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 27400
Aug 07 04:10:22 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.2@KRBTEST.COM
I20260807 04:10:22.967638 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:22.952290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.18.193.2:41849)
0807 04:10:22.952529 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:22.952536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:22.952672 (+ 136us) server_negotiation.cc:407] Connection header received
0807 04:10:22.952906 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:22.952911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:22.952990 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:22.953105 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:22.954327 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.955048 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.956330 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:22.956956 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:22.957098 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:22.961693 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:22.961710 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:22.961726 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:22.961767 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:22.964869 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:22.965690 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:22.965698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:22.965700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:22.965839 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:22.966249 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:22.966252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:22.966254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:22.966611 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:22.966839 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:22.967126 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 04:10:22.967298 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 04:10:22.968382 27535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:23.106612 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 27400
I20260807 04:10:23.132413 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 27545
Aug 07 04:10:23 dist-test-slave-8n49 krb5kdc[27129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.193.3@KRBTEST.COM
I20260807 04:10:23.150286 27827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:23.133343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.18.193.3:45489)
0807 04:10:23.133531 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:23.133537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:23.133572 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:10:23.133745 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:23.133749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:23.133827 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:23.133917 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:23.135213 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:23.135967 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:23.137405 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:23.137858 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:23.137975 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:23.143896 (+ 5921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:23.143914 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:23.143931 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:23.143974 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:23.147065 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:23.147958 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:23.147965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:23.147968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:23.148132 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:23.148690 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:23.148694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:23.148696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:23.149097 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:23.149349 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:23.149799 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 04:10:23.150008 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:10:23.151139 27685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:23.280682 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 27545
I20260807 04:10:23.305583 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 27142
I20260807 04:10:23.306854 27218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:23.431383 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 27142
2026-08-07T04:10:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7321 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:10:23 dist-test-slave-8n49 krb5kdc[27844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:10:23 dist-test-slave-8n49 krb5kdc[27844](info): set up 2 sockets
Aug 07 04:10:23 dist-test-slave-8n49 krb5kdc[27844](info): commencing operation
krb5kdc: starting...
W20260807 04:10:24.587404 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:10:24 dist-test-slave-8n49 krb5kdc[27844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:10:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:10:24.865806 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35115
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:35115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:25.212939 27858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:25.213269 27858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:25.213426 27858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:25.221109 27858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:25.221227 27858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:25.221276 27858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:25.221369 27858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:25.221450 27858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:25.231966 27858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:35115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:25.233402 27858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:25.234776 27858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:25.247632 27864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:25.247701 27863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:25.247885 27858 server_base.cc:1034] running on GCE node
W20260807 04:10:25.247665 27866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:25.249420 27858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:25.251072 27858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:25.252398 27858 hybrid_clock.cc:648] HybridClock initialized: now 1786075825252296 us; error 45 us; skew 500 ppm
I20260807 04:10:25.252851 27858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T04:10:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1892 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:10:25 dist-test-slave-8n49 krb5kdc[27873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:10:25 dist-test-slave-8n49 krb5kdc[27873](info): set up 2 sockets
Aug 07 04:10:25 dist-test-slave-8n49 krb5kdc[27873](info): commencing operation
krb5kdc: starting...
W20260807 04:10:26.495011 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:10:26 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:10:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:10:26.774101 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37945
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:37945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:27.137064 27886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:27.137409 27886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:27.137607 27886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:27.145684 27886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:27.145816 27886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:27.145907 27886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:27.146028 27886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:27.146098 27886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:27.159679 27886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:37945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:27.161520 27886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:27.163300 27886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:27.176013 27898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:27.176143 27895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:27.176050 27894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:27.182528 27886 server_base.cc:1034] running on GCE node
I20260807 04:10:27.183495 27886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:27.185138 27886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:27.186401 27886 hybrid_clock.cc:648] HybridClock initialized: now 1786075827186321 us; error 71 us; skew 500 ppm
I20260807 04:10:27.186820 27886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:27 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075827, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:27.190429 27886 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:10:27.192435 27886 webserver.cc:460] Webserver started at http://127.18.193.62:33735/ using document root <none> and password file <none>
I20260807 04:10:27.193250 27886 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:27.193389 27886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:27.193742 27886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:27.197027 27886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "7c2fc807705d4ea4bd40e3503f21f882"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "b052b976bd73f24b6029d4c05b0dbd0a"
server_key_iv: "02b0b46a70e25e0c1e98e2d959c6c818"
server_key_version: "encryptionkey@0"
I20260807 04:10:27.197832 27886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "7c2fc807705d4ea4bd40e3503f21f882"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "b052b976bd73f24b6029d4c05b0dbd0a"
server_key_iv: "02b0b46a70e25e0c1e98e2d959c6c818"
server_key_version: "encryptionkey@0"
I20260807 04:10:27.203411 27886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 04:10:27.207681 27905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:27.209292 27886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:27.209506 27886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "7c2fc807705d4ea4bd40e3503f21f882"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "b052b976bd73f24b6029d4c05b0dbd0a"
server_key_iv: "02b0b46a70e25e0c1e98e2d959c6c818"
server_key_version: "encryptionkey@0"
I20260807 04:10:27.209724 27886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:27.239920 27886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:27.245077 27886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:27.245440 27886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:27.267226 27958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:37945 every 8 connection(s)
I20260807 04:10:27.267226 27886 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:37945
I20260807 04:10:27.268808 27886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:27.272692 27959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:27.277868 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 27886
I20260807 04:10:27.278106 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:27.278774 19204 external_mini_cluster.cc:1428] Setting key 9a78935c9759d8614a03feea71279720
I20260807 04:10:27.283874 27959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Bootstrap starting.
I20260807 04:10:27.286942 27959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:27.287951 27959 log.cc:824] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:27.290843 27959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: No bootstrap required, opened a new log
Aug 07 04:10:27 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:27.299181 27959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:27.299587 27959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:27.299719 27959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2fc807705d4ea4bd40e3503f21f882, State: Initialized, Role: FOLLOWER
I20260807 04:10:27.300474 27959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:27.300777 27959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:27.300931 27959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:27.301118 27959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:27.302156 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:27.280865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.18.193.62:37945)
0807 04:10:27.281835 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:27.281849 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:27.281907 (+ 58us) server_negotiation.cc:407] Connection header received
0807 04:10:27.283641 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:27.283664 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:27.284089 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:27.284466 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:27.286032 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:27.287418 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:27.288749 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:27.289428 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:27.289622 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:27.293833 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:27.293840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:27.293854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:27.293890 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:27.297308 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:27.298105 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:27.298115 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:27.298122 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:27.298311 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:27.298732 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:27.298736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:27.298738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:27.299309 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:27.299632 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:27.300146 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 04:10:27.300671 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":520,"threads_started":1}
I20260807 04:10:27.304344 27959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:27.304966 27959 leader_election.cc:302] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c2fc807705d4ea4bd40e3503f21f882; no voters:
I20260807 04:10:27.305591 27959 leader_election.cc:288] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:27.305747 27964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:27.306452 27964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 LEADER]: Becoming Leader. State: Replica: 7c2fc807705d4ea4bd40e3503f21f882, State: Running, Role: LEADER
I20260807 04:10:27.306985 27964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:27.307636 27959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:27.309906 27965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c2fc807705d4ea4bd40e3503f21f882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } } }
I20260807 04:10:27.310366 27965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:27.310702 27966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2fc807705d4ea4bd40e3503f21f882. Latest consensus state: current_term: 1 leader_uuid: "7c2fc807705d4ea4bd40e3503f21f882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } } }
I20260807 04:10:27.311034 27966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:27.312466 27969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:27.318053 27969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:27.327986 27969 catalog_manager.cc:1347] Generated new cluster ID: 3d853da542c6427ea114eba38080d83c
I20260807 04:10:27.328122 27969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:27.345125 27969 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:27.346561 27969 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:27.359068 27969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Generated new TSK 0
I20260807 04:10:27.360013 27969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:10:27.433490 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37945
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:27.785356 27986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:27.785692 27986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:27.785863 27986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:27.794035 27986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:27.794209 27986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:27.794399 27986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:27.805291 27986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:27.806718 27986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:27.808254 27986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:27.822664 27992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:27.823104 27991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:27.823599 27994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:27.824157 27986 server_base.cc:1034] running on GCE node
I20260807 04:10:27.824888 27986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:27.825918 27986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:27.827167 27986 hybrid_clock.cc:648] HybridClock initialized: now 1786075827827119 us; error 63 us; skew 500 ppm
I20260807 04:10:27.827505 27986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:27 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075827, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:27.831020 27986 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:10:27.832646 27986 webserver.cc:460] Webserver started at http://127.18.193.1:37625/ using document root <none> and password file <none>
I20260807 04:10:27.833365 27986 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:27.833477 27986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:27.833736 27986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:27.836977 27986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "d9a53731b87448098eabc80e6b77656d"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "4a2cb40ab0859b59323ee2c9bc0e59fd"
server_key_iv: "acf64c193c10735968412d088091336f"
server_key_version: "encryptionkey@0"
I20260807 04:10:27.837779 27986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "d9a53731b87448098eabc80e6b77656d"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "4a2cb40ab0859b59323ee2c9bc0e59fd"
server_key_iv: "acf64c193c10735968412d088091336f"
server_key_version: "encryptionkey@0"
I20260807 04:10:27.843336 27986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:10:27.847138 28001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:27.848754 27986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:27.848933 27986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "d9a53731b87448098eabc80e6b77656d"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "4a2cb40ab0859b59323ee2c9bc0e59fd"
server_key_iv: "acf64c193c10735968412d088091336f"
server_key_version: "encryptionkey@0"
I20260807 04:10:27.849161 27986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:27.869140 27986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:27.873642 27986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:27.873966 27986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:27.875007 27986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:27.876410 27986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:27.876483 27986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:27.876637 27986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:27.876696 27986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:27.911391 27986 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:37299
I20260807 04:10:27.911438 28116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:37299 every 8 connection(s)
I20260807 04:10:27.912835 27986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:27.915212 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 27986
I20260807 04:10:27.915422 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:10:27.916290 19204 external_mini_cluster.cc:1428] Setting key 60069e209aafb1731814c8e3962473d7
Aug 07 04:10:27 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075827, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:27.935243 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:27.915772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:60901 (local address 127.18.193.62:37945)
0807 04:10:27.916640 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:27.916645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:27.917794 (+ 1149us) server_negotiation.cc:407] Connection header received
0807 04:10:27.919987 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:27.919996 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:27.920092 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:27.920216 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:27.921877 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:27.923047 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:27.924732 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:27.925478 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:27.925644 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:27.929471 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:27.929476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:27.929480 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:27.929512 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:27.932015 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:27.932886 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:27.932896 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:27.932898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:27.933014 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:27.933458 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:27.933467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:27.933469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:27.933732 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:27.933892 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:27.934617 (+ 725us) server_negotiation.cc:299] Negotiation successful
0807 04:10:27.934849 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":272,"threads_started":1}
I20260807 04:10:27.936246 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:27.916493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:60901)
0807 04:10:27.917613 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:27.917651 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:27.919699 (+ 2048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:27.920435 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:27.920443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:27.921116 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:27.921696 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:27.921715 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:27.923244 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:27.923249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:27.924513 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:27.924527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:27.925742 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:27.925747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:27.925964 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:27.926721 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:27.926749 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:27.929215 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:27.932183 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:27.932193 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:27.932210 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:27.932749 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:27.933139 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:27.933144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:27.933148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:27.933333 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:27.933866 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:27.933874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:27.934232 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 04:10:27.934503 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 04:10:27.935060 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"thread_start_us":406,"threads_started":1}
I20260807 04:10:27.938045 28117 heartbeater.cc:347] Connected to a master server at 127.18.193.62:37945
I20260807 04:10:27.938447 28117 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:27.939450 28117 heartbeater.cc:511] Master 127.18.193.62:37945 requested a full tablet report, sending...
I20260807 04:10:27.941911 27923 ts_manager.cc:194] Registered new tserver with Master: d9a53731b87448098eabc80e6b77656d (127.18.193.1:37299)
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
I20260807 04:10:27.943478 27923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:60901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:10:27.979813 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37945
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:28.319022 28123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:28.319389 28123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:28.319582 28123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:28.327332 28123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:28.327520 28123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:28.327705 28123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:28.337939 28123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:28.339929 28123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:28.341395 28123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:28.355015 28130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:28.354972 28131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:28.355058 28133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:28.356305 28123 server_base.cc:1034] running on GCE node
I20260807 04:10:28.356993 28123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:28.358001 28123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:28.359244 28123 hybrid_clock.cc:648] HybridClock initialized: now 1786075828359214 us; error 65 us; skew 500 ppm
I20260807 04:10:28.359566 28123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:28 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075828, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:28.362882 28123 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:10:28.364432 28123 webserver.cc:460] Webserver started at http://127.18.193.2:45407/ using document root <none> and password file <none>
I20260807 04:10:28.365224 28123 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:28.365375 28123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:28.365710 28123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:28.368686 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "3d8ab53dc68048c281d0d2045483313a"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "c120c96e966e0db125924ea5139eee4c"
server_key_iv: "041a2cb5467f7313ffd1a34f3a6e2dab"
server_key_version: "encryptionkey@0"
I20260807 04:10:28.369585 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "3d8ab53dc68048c281d0d2045483313a"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "c120c96e966e0db125924ea5139eee4c"
server_key_iv: "041a2cb5467f7313ffd1a34f3a6e2dab"
server_key_version: "encryptionkey@0"
I20260807 04:10:28.374854 28123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:10:28.378407 28140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:28.379741 28123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:10:28.379930 28123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "3d8ab53dc68048c281d0d2045483313a"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "c120c96e966e0db125924ea5139eee4c"
server_key_iv: "041a2cb5467f7313ffd1a34f3a6e2dab"
server_key_version: "encryptionkey@0"
I20260807 04:10:28.380149 28123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:28.403358 28123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:28.407732 28123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:28.408032 28123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:28.409143 28123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:28.410496 28123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:28.410609 28123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:28.410784 28123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:28.410862 28123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:28.446543 28123 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:43019
I20260807 04:10:28.446592 28258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:43019 every 8 connection(s)
I20260807 04:10:28.447901 28123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:28.456785 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 28123
I20260807 04:10:28.456993 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:10:28.457626 19204 external_mini_cluster.cc:1428] Setting key eb0ae344bc44279b0fb8648f39b4c466
Aug 07 04:10:28 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075828, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:28.470686 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:28.450813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:56919 (local address 127.18.193.62:37945)
0807 04:10:28.451344 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:28.451351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:28.452593 (+ 1242us) server_negotiation.cc:407] Connection header received
0807 04:10:28.454608 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:28.454613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:28.454692 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:28.454798 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:28.456398 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:28.457614 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:28.459487 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:28.460010 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:28.460113 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:28.464495 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:28.464501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:28.464504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:28.464539 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:28.467319 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:28.468112 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:28.468118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:28.468122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:28.468262 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:28.468808 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:28.468813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:28.468815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:28.469093 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:28.469256 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:28.470031 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 04:10:28.470287 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":263,"threads_started":1}
I20260807 04:10:28.471717 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:28.451544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:56919)
0807 04:10:28.452382 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:28.452425 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:28.454306 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:28.455013 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:28.455021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:28.455580 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:28.456143 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:28.456165 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:28.457844 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:28.457854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:28.459330 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:28.459348 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:28.460257 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:28.460264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:28.460622 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:28.461419 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:28.461456 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:28.464219 (+ 2763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:28.467498 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:28.467515 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:28.467534 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:28.467980 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:28.468390 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:28.468395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:28.468398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:28.468679 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:28.469229 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:28.469239 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:28.469649 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 04:10:28.469922 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 04:10:28.470481 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":290,"threads_started":1}
I20260807 04:10:28.473456 28259 heartbeater.cc:347] Connected to a master server at 127.18.193.62:37945
I20260807 04:10:28.473801 28259 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:28.474537 28259 heartbeater.cc:511] Master 127.18.193.62:37945 requested a full tablet report, sending...
I20260807 04:10:28.476138 27923 ts_manager.cc:194] Registered new tserver with Master: 3d8ab53dc68048c281d0d2045483313a (127.18.193.2:43019)
I20260807 04:10:28.477033 27923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:56919
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:10:28.521476 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37945
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:28.859310 28265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:28.859619 28265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:28.859768 28265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:28.867571 28265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:28.867708 28265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:28.867895 28265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:28.878584 28265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:28.879968 28265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:28.881322 28265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:28.894853 28273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:28.894981 28270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:28.894987 28271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:28.896521 28265 server_base.cc:1034] running on GCE node
I20260807 04:10:28.897207 28265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:28.898255 28265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:28.899492 28265 hybrid_clock.cc:648] HybridClock initialized: now 1786075828899426 us; error 55 us; skew 500 ppm
I20260807 04:10:28.899825 28265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:28 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075828, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:28.903223 28265 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:10:28.904857 28265 webserver.cc:460] Webserver started at http://127.18.193.3:35875/ using document root <none> and password file <none>
I20260807 04:10:28.905599 28265 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:28.905707 28265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:28.905985 28265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:28.909343 28265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "eb5acc06609e4925b3b21200767735f2"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "5262061d9857778b25693e4d23f8b111"
server_key_iv: "748cd7442e4c927153c0c47df7b1aa2e"
server_key_version: "encryptionkey@0"
I20260807 04:10:28.910163 28265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "eb5acc06609e4925b3b21200767735f2"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "5262061d9857778b25693e4d23f8b111"
server_key_iv: "748cd7442e4c927153c0c47df7b1aa2e"
server_key_version: "encryptionkey@0"
I20260807 04:10:28.916355 28265 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 04:10:28.920146 28281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:28.921690 28265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:28.921862 28265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "eb5acc06609e4925b3b21200767735f2"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "5262061d9857778b25693e4d23f8b111"
server_key_iv: "748cd7442e4c927153c0c47df7b1aa2e"
server_key_version: "encryptionkey@0"
I20260807 04:10:28.922088 28265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:28.946741 28117 heartbeater.cc:503] Master 127.18.193.62:37945 was elected leader, sending a full tablet report...
I20260807 04:10:28.947057 28265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:28.951988 28265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:28.952334 28265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:28.953500 28265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:28.954957 28265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:28.955036 28265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:28.955209 28265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:28.955260 28265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:28.990531 28265 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:40157
I20260807 04:10:28.990630 28398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:40157 every 8 connection(s)
I20260807 04:10:28.991856 28265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:28.998566 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 28265
I20260807 04:10:28.998757 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:10:28.999336 19204 external_mini_cluster.cc:1428] Setting key 78482c37b27d5da10f43146709d29b3b
Aug 07 04:10:29 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075828, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:29.013288 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:28.994599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:54001 (local address 127.18.193.62:37945)
0807 04:10:28.995110 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:28.995115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:28.996037 (+ 922us) server_negotiation.cc:407] Connection header received
0807 04:10:28.997935 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:28.997939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:28.998019 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:28.998133 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:29.000267 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.001197 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.002952 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.003376 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.003515 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:29.007107 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:29.007112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:29.007115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:29.007157 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:29.010066 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.010804 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.010808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.010810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.010945 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.011374 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.011378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.011381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.011691 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:29.011825 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:29.012729 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 04:10:29.012916 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":249,"threads_started":1}
I20260807 04:10:29.014554 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:28.995121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:54001)
0807 04:10:28.995855 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:28.995902 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:28.997667 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:28.998416 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:28.998427 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:28.999190 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:29.000078 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:29.000101 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.001346 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.001352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:29.002774 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:29.002794 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.003655 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.003663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:29.003894 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:29.004627 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:29.004652 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:29.006893 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:29.010216 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:29.010223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:29.010237 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:29.010688 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:29.011054 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:29.011057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:29.011059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:29.011244 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:29.011817 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:29.011823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:29.012239 (+ 416us) client_negotiation.cc:769] Sending connection context
0807 04:10:29.012645 (+ 406us) client_negotiation.cc:240] Negotiation successful
0807 04:10:29.013122 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":302,"threads_started":1}
I20260807 04:10:29.015966 28399 heartbeater.cc:347] Connected to a master server at 127.18.193.62:37945
I20260807 04:10:29.016253 28399 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:29.017309 28399 heartbeater.cc:511] Master 127.18.193.62:37945 requested a full tablet report, sending...
I20260807 04:10:29.018824 27923 ts_manager.cc:194] Registered new tserver with Master: eb5acc06609e4925b3b21200767735f2 (127.18.193.3:40157)
I20260807 04:10:29.019665 27923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:54001
I20260807 04:10:29.027751 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab.
Entry for principal oryx/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab.
WARNING: no policy specified for oryx/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab.
Entry for principal oryx/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab.
WARNING: no policy specified for oryx/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab.
Entry for principal oryx/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab.
I20260807 04:10:29.200536 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 27986
Aug 07 04:10:29 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:10:29.217638 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:29.201874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.18.193.1:37299)
0807 04:10:29.202631 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:29.202638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:29.202666 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:10:29.202729 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:29.202736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:29.202945 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:29.203076 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:29.204409 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.205481 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.206802 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.207362 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.207457 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:29.211783 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:29.211795 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:29.211810 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:29.211843 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:29.214910 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.215537 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.215543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.215546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.215670 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.215991 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.215994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.215995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.216330 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:29.216618 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:29.216865 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 04:10:29.217126 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":424,"threads_started":1}
I20260807 04:10:29.218943 28112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:29.302430 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 27986
I20260807 04:10:29.322160 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28123
Aug 07 04:10:29 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:10:29.340054 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:29.323315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41498 (local address 127.18.193.2:43019)
0807 04:10:29.324028 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:29.324036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:29.324068 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:10:29.324133 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:29.324140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:29.324365 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:29.324603 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:29.326047 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.327146 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.328437 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.329188 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.329335 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:29.333724 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:29.333736 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:29.333752 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:29.333796 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:29.337047 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.337769 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.337783 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.337785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.337921 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.338314 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.338319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.338323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.338701 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:29.339031 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:29.339384 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 04:10:29.339684 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":330,"threads_started":1}
I20260807 04:10:29.341516 28254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:29.429658 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 28123
I20260807 04:10:29.447432 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28265
Aug 07 04:10:29 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:10:29.466323 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:29.448524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.18.193.3:40157)
0807 04:10:29.449191 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:29.449198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:29.449231 (+ 33us) server_negotiation.cc:407] Connection header received
0807 04:10:29.449295 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:29.449302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:29.449522 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:29.449722 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:29.451107 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.452130 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.453613 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:29.454194 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:29.454355 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:29.459542 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:29.459558 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:29.459573 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:29.459651 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:29.463086 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.463903 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.463911 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.463913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.464066 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:29.464547 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:29.464604 (+ 57us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:29.464609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:29.465062 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:29.465324 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:29.465619 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 04:10:29.465945 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":304,"threads_started":1}
I20260807 04:10:29.467868 28393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:29.553238 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 28265
I20260807 04:10:29.572661 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 27886
I20260807 04:10:29.574105 27954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:29.681782 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 27886
I20260807 04:10:29.705842 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37945
--webserver_interface=127.18.193.62
--webserver_port=33735
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:37945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:30.047331 28418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:30.047655 28418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:30.047748 28418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:30.055168 28418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:30.055291 28418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:30.055341 28418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:30.055390 28418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:30.055437 28418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:30.065565 28418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:37945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=33735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:30.066898 28418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:30.068189 28418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:30.080287 28424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:30.080264 28427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:30.080470 28425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:30.081921 28418 server_base.cc:1034] running on GCE node
I20260807 04:10:30.082717 28418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:30.084164 28418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:30.085378 28418 hybrid_clock.cc:648] HybridClock initialized: now 1786075830085318 us; error 52 us; skew 500 ppm
I20260807 04:10:30.085752 28418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:30 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075830, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:30.088925 28418 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:10:30.090585 28418 webserver.cc:460] Webserver started at http://127.18.193.62:33735/ using document root <none> and password file <none>
I20260807 04:10:30.091315 28418 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:30.091485 28418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:30.097164 28418 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:10:30.100183 28435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:30.101706 28418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:30.101907 28418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "7c2fc807705d4ea4bd40e3503f21f882"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "b052b976bd73f24b6029d4c05b0dbd0a"
server_key_iv: "02b0b46a70e25e0c1e98e2d959c6c818"
server_key_version: "encryptionkey@0"
I20260807 04:10:30.102954 28418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:30.125002 28418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:30.129621 28418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:30.129933 28418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:30.152098 28418 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:37945
I20260807 04:10:30.152117 28489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:37945 every 8 connection(s)
I20260807 04:10:30.154014 28418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:30.161270 28490 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:10:30.163266 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 28418
I20260807 04:10:30.164284 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:37299
--local_ip_for_outbound_sockets=127.18.193.1
--tserver_master_addrs=127.18.193.62:37945
--webserver_port=37625
--webserver_interface=127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:10:30.165767 28490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Bootstrap starting.
I20260807 04:10:30.188344 28490 log.cc:824] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:30.198104 28490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:10:30.198681 28490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Bootstrap complete.
I20260807 04:10:30.208276 28490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:30.209245 28490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2fc807705d4ea4bd40e3503f21f882, State: Initialized, Role: FOLLOWER
I20260807 04:10:30.209877 28490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:30.210119 28490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:30.210244 28490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:30.210433 28490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:10:30.214423 28490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:30.214956 28490 leader_election.cc:302] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c2fc807705d4ea4bd40e3503f21f882; no voters:
I20260807 04:10:30.215850 28490 leader_election.cc:288] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:10:30.216028 28494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:10:30.217401 28494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [term 2 LEADER]: Becoming Leader. State: Replica: 7c2fc807705d4ea4bd40e3503f21f882, State: Running, Role: LEADER
I20260807 04:10:30.218173 28490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:30.218058 28494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } }
I20260807 04:10:30.221374 28496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2fc807705d4ea4bd40e3503f21f882. Latest consensus state: current_term: 2 leader_uuid: "7c2fc807705d4ea4bd40e3503f21f882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } } }
I20260807 04:10:30.221882 28496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:30.222544 28495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7c2fc807705d4ea4bd40e3503f21f882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2fc807705d4ea4bd40e3503f21f882" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37945 } } }
I20260807 04:10:30.224699 28495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882 [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:30.232623 28501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:30.238663 28501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:30.240639 28501 catalog_manager.cc:1259] Loaded cluster ID: 3d853da542c6427ea114eba38080d83c
I20260807 04:10:30.240770 28501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:30.246065 28501 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:30.248175 28501 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7c2fc807705d4ea4bd40e3503f21f882: Loaded TSK: 0
I20260807 04:10:30.250267 28501 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:10:30.571300 28492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:30.571672 28492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:30.571835 28492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:30.579624 28492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:30.579800 28492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:30.579977 28492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:30.590369 28492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.1.keytab
--principal=oryx/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:37299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=37625
--tserver_master_addrs=127.18.193.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:30.591832 28492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:30.593353 28492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:30.610431 28521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:30.610333 28519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:30.610378 28518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:30.611466 28492 server_base.cc:1034] running on GCE node
I20260807 04:10:30.612460 28492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:30.613715 28492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:30.615018 28492 hybrid_clock.cc:648] HybridClock initialized: now 1786075830614932 us; error 67 us; skew 500 ppm
I20260807 04:10:30.615388 28492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:30 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075830, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:30.622550 28492 init.cc:377] Logged in from keytab as oryx/127.18.193.1@KRBTEST.COM (short username oryx)
I20260807 04:10:30.624445 28492 webserver.cc:460] Webserver started at http://127.18.193.1:37625/ using document root <none> and password file <none>
I20260807 04:10:30.625229 28492 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:30.625383 28492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:30.631201 28492 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 04:10:30.634426 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:30.635957 28492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:10:30.636157 28492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "d9a53731b87448098eabc80e6b77656d"
format_stamp: "Formatted at 2026-08-07 04:10:27 on dist-test-slave-8n49"
server_key: "4a2cb40ab0859b59323ee2c9bc0e59fd"
server_key_iv: "acf64c193c10735968412d088091336f"
server_key_version: "encryptionkey@0"
I20260807 04:10:30.637288 28492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:30.672673 28492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:30.677421 28492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:30.677733 28492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:30.678850 28492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:30.680236 28492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:30.680353 28492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:30.680469 28492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:30.680588 28492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:30.714767 28492 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:37299
I20260807 04:10:30.714860 28645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:37299 every 8 connection(s)
I20260807 04:10:30.716300 28492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:30.726543 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 28492
I20260807 04:10:30.727234 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:43019
--local_ip_for_outbound_sockets=127.18.193.2
--tserver_master_addrs=127.18.193.62:37945
--webserver_port=45407
--webserver_interface=127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:10:30 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:30 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:10:31.103622 28650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:31.103928 28650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:31.104081 28650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:31.111783 28650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:31.111908 28650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:31.112108 28650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:31.122694 28650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.2.keytab
--principal=oryx/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:43019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=45407
--tserver_master_addrs=127.18.193.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:31.124116 28650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:31.125541 28650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:31.141731 28657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:31.141746 28659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:31.142078 28656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:31.143795 28650 server_base.cc:1034] running on GCE node
I20260807 04:10:31.144539 28650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:31.145826 28650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:31.147079 28650 hybrid_clock.cc:648] HybridClock initialized: now 1786075831147029 us; error 44 us; skew 500 ppm
I20260807 04:10:31.147433 28650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:31 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075831, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:31.151156 28650 init.cc:377] Logged in from keytab as oryx/127.18.193.2@KRBTEST.COM (short username oryx)
I20260807 04:10:31.153510 28650 webserver.cc:460] Webserver started at http://127.18.193.2:45407/ using document root <none> and password file <none>
I20260807 04:10:31.154587 28650 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:31.154769 28650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:31.162510 28650 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:10:31.166519 28666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:31.168309 28650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 04:10:31.168602 28650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "3d8ab53dc68048c281d0d2045483313a"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "c120c96e966e0db125924ea5139eee4c"
server_key_iv: "041a2cb5467f7313ffd1a34f3a6e2dab"
server_key_version: "encryptionkey@0"
I20260807 04:10:31.169862 28650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:31.184896 28650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:31.189687 28650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:31.189975 28650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:31.191022 28650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:31.192466 28650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:31.192600 28650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:31.192739 28650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:31.192824 28650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:31.235162 28650 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:43019
I20260807 04:10:31.235320 28783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:43019 every 8 connection(s)
I20260807 04:10:31.236812 28650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:31.245831 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 28650
I20260807 04:10:31.246686 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:40157
--local_ip_for_outbound_sockets=127.18.193.3
--tserver_master_addrs=127.18.193.62:37945
--webserver_port=35875
--webserver_interface=127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:10:31 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:31 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:10:31.961292 28649 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.228s user 0.001s sys 0.014s
W20260807 04:10:31.963351 28648 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:30.720439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:40785 (local address 127.18.193.62:37945)
0807 04:10:30.721634 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:30.721651 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:30.722027 (+ 376us) server_negotiation.cc:407] Connection header received
0807 04:10:30.724169 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:30.724181 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:30.724492 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:30.724925 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:30.726814 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:30.728207 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:30.730707 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:30.731319 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:30.731505 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:31.962578 (+1231073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:40785: BlockingRecv error: recv got EOF from 127.18.193.1:40785 (error 108)
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":595,"threads_started":1}
I20260807 04:10:31.963356 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:30.720511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:40785)
0807 04:10:30.721841 (+ 1330us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:30.721881 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:30.723614 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:30.725093 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:30.725105 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:30.725734 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:30.726501 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:30.726543 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:30.728401 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:30.728409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:30.730557 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:30.730575 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:30.731573 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:30.731584 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:30.731899 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:30.732783 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:30.732806 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:31.962151 (+1229345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1045,"thread_start_us":805,"threads_started":1}
W20260807 04:10:31.964640 28646 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:37945 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.193.62:37945: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Aug 07 04:10:31 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:31 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:31.974699 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:31.965373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:49045)
0807 04:10:31.965722 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:31.965747 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:31.965887 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:31.966301 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:31.966306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:31.966785 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:31.966949 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:31.966957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.967990 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.967995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:31.969110 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:31.969123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.969906 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.969912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:31.970081 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:31.970732 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:31.970749 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:31.974370 (+ 3621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194}
W20260807 04:10:31.974843 28648 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:31.965744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:49045 (local address 127.18.193.62:37945)
0807 04:10:31.965905 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:31.965910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:31.965932 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:10:31.966003 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:31.966011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:31.966101 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:31.966214 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:31.967104 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.967839 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.969252 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.969736 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.969877 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:31.974581 (+ 4704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:49045: BlockingRecv error: recv got EOF from 127.18.193.1:49045 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 04:10:31 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:31 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:31.984385 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:31.975824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:60407)
0807 04:10:31.976142 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:31.976163 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:31.976268 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:31.976818 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:31.976821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:31.977091 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:31.977234 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:31.977241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.978280 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.978286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:31.979228 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:31.979240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.979958 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.979963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:31.980184 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:31.980822 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:31.980838 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:31.984088 (+ 3250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154}
W20260807 04:10:31.984627 28648 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:31.975985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:60407 (local address 127.18.193.62:37945)
0807 04:10:31.976143 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:31.976148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:31.976213 (+ 65us) server_negotiation.cc:407] Connection header received
0807 04:10:31.976391 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:31.976396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:31.976463 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:31.976667 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:31.977373 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.978134 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.979367 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.979789 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.979949 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:31.984302 (+ 4353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:60407: BlockingRecv error: recv got EOF from 127.18.193.1:60407 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 04:10:31.984341 28788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:31.984936 28788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:31.985145 28646 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:10:31.985188 28788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:32.004843 28788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:32.005120 28788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:32.005450 28788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:32.031256 28788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/krb5kdc/oryx_127.18.193.3.keytab
--principal=oryx/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:40157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=35875
--tserver_master_addrs=127.18.193.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:32.033937 28788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:32.036468 28788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:32.056831 28795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:32.056875 28794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:32.061810 28798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:32.062870 28788 server_base.cc:1034] running on GCE node
I20260807 04:10:32.064046 28788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:32.065907 28788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:32.067237 28788 hybrid_clock.cc:648] HybridClock initialized: now 1786075832067158 us; error 71 us; skew 500 ppm
I20260807 04:10:32.067807 28788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075832, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:32.073333 28788 init.cc:377] Logged in from keytab as oryx/127.18.193.3@KRBTEST.COM (short username oryx)
I20260807 04:10:32.075390 28788 webserver.cc:460] Webserver started at http://127.18.193.3:35875/ using document root <none> and password file <none>
I20260807 04:10:32.076180 28788 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:32.076351 28788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:32.083036 28788 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 04:10:32.086390 28805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:32.088088 28788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:10:32.088279 28788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "eb5acc06609e4925b3b21200767735f2"
format_stamp: "Formatted at 2026-08-07 04:10:28 on dist-test-slave-8n49"
server_key: "5262061d9857778b25693e4d23f8b111"
server_key_iv: "748cd7442e4c927153c0c47df7b1aa2e"
server_key_version: "encryptionkey@0"
I20260807 04:10:32.089378 28788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:32.116730 28788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:32.121592 28788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:32.121884 28788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:32.122977 28788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:32.124449 28788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:32.124605 28788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:32.124744 28788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:32.124831 28788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:32.177628 28788 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:40157
I20260807 04:10:32.177881 28921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:40157 every 8 connection(s)
I20260807 04:10:32.179209 28788 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:32.180572 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 28788
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:32.207083 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:32.181651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.18.193.62:37945)
0807 04:10:32.181961 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:32.181972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:32.182849 (+ 877us) server_negotiation.cc:407] Connection header received
0807 04:10:32.183142 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:32.183148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:32.183232 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:32.183340 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:32.186793 (+ 3453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.187849 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.190653 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.191302 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.191432 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.197589 (+ 6157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:32.197603 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:32.197628 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:32.197686 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:32.202309 (+ 4623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:32.203562 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:32.203573 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:32.203583 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:32.203810 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:32.204351 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:32.204356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:32.204361 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:32.205144 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:32.205480 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:32.206114 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 04:10:32.206617 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260807 04:10:32.514089 28787 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.258s user 0.004s sys 0.010s
W20260807 04:10:32.515718 28786 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:31.239794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:38607 (local address 127.18.193.62:37945)
0807 04:10:31.240393 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:31.240399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:31.241752 (+ 1353us) server_negotiation.cc:407] Connection header received
0807 04:10:31.245841 (+ 4089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:31.245847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:31.245949 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:31.246093 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:31.248716 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.250517 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.253253 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.254287 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.254485 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.515343 (+1260858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:38607: BlockingRecv error: recv got EOF from 127.18.193.2:38607 (error 108)
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":333,"threads_started":1}
I20260807 04:10:32.516310 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:31.240167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:38607)
0807 04:10:31.241468 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:31.241547 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:31.245365 (+ 3818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:31.246618 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:31.246632 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:31.247482 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:31.248368 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:31.248402 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.250754 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.250763 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:31.253048 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:31.253075 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:31.254576 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:31.254586 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:31.254919 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:31.255966 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:31.256002 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:32.515049 (+1259047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":464,"threads_started":1}
W20260807 04:10:32.517252 28784 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:37945 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.193.62:37945: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:10:32.529083 28786 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:32.518141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:54749 (local address 127.18.193.62:37945)
0807 04:10:32.518340 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:32.518346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:32.518364 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:10:32.518533 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:32.518559 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:32.518645 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:32.518761 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:32.519491 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.520358 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.521704 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.522157 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.522324 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.528908 (+ 6584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:54749: BlockingRecv error: recv got EOF from 127.18.193.2:54749 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:10:32.529119 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:32.518028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:54749)
0807 04:10:32.518231 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:32.518254 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:32.518375 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:32.518791 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:32.518796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:32.519179 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:32.519342 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.519351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.520519 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.520527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.521550 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.521562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.522362 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.522368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.522573 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.523335 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:32.523353 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:32.528622 (+ 5269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:32.542105 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:32.530092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:46831)
0807 04:10:32.530502 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:32.530522 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:32.530638 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:32.531243 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:32.531248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:32.531621 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:32.531782 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.531791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.534092 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.534097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.535163 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.535176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.536538 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.536542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.536720 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.537366 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:32.537384 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:32.541760 (+ 4376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282}
W20260807 04:10:32.542634 28784 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:10:32.543326 28786 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:32.530547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:46831 (local address 127.18.193.62:37945)
0807 04:10:32.530795 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:32.530804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:32.530831 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:10:32.530911 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:32.530920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:32.531068 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:32.531212 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:32.531943 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.533827 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.535327 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.536350 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.536543 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.542886 (+ 6343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:46831: BlockingRecv error: recv got EOF from 127.18.193.2:46831 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:32 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:32.998086 28928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:32.985999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:35537)
0807 04:10:32.986633 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:32.986650 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:32.986769 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:32.987274 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:32.987277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:32.987720 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:32.987867 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.987875 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.989872 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.989876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.991022 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.991036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.992513 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.992520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.992775 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.993749 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:32.993773 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:32.997834 (+ 4061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":382,"threads_started":1}
W20260807 04:10:32.998804 28786 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:32.986132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:35537 (local address 127.18.193.62:37945)
0807 04:10:32.986402 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:32.986413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:32.986748 (+ 335us) server_negotiation.cc:407] Connection header received
0807 04:10:32.986916 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:32.986925 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:32.987065 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:32.987229 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:32.988029 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.989680 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.991207 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.992230 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.992455 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.998235 (+ 5780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:35537: BlockingRecv error: recv got EOF from 127.18.193.1:35537 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 04:10:33.307015 28926 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.113s user 0.008s sys 0.008s
W20260807 04:10:33.308467 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:32.183397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:35517 (local address 127.18.193.62:37945)
0807 04:10:32.183925 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:32.183931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:32.185022 (+ 1091us) server_negotiation.cc:407] Connection header received
0807 04:10:32.187085 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:32.187094 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:32.187236 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:32.187401 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:32.189097 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.189882 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.191867 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.192346 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.192492 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:33.308113 (+1115621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:35517: BlockingRecv error: recv got EOF from 127.18.193.3:35517 (error 108)
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":263,"threads_started":1}
I20260807 04:10:33.309183 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:32.183986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:35517)
0807 04:10:32.184882 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:32.184926 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:32.186749 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:32.187653 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:32.187664 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:32.188272 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:32.188917 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.188935 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.190028 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.190040 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.191719 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:32.191731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:32.192754 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:32.192767 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:32.193279 (+ 512us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:32.194173 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:32.194196 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:33.307892 (+1113696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":380,"threads_started":1}
W20260807 04:10:33.310222 28922 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:37945 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.193.62:37945: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Aug 07 04:10:33 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:33 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:33.319824 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:33.310969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:58957)
0807 04:10:33.311166 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:33.311182 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:33.311275 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:33.311569 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:33.311572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:33.311911 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:33.312085 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:33.312094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.313341 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.313346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:33.314343 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:33.314357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.315085 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.315100 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:33.315227 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:33.315900 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:33.315918 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:33.319595 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 04:10:33.320150 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:33.311013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:58957 (local address 127.18.193.62:37945)
0807 04:10:33.311243 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:33.311248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:33.311264 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:10:33.311379 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:33.311383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:33.311443 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:33.311524 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:33.312262 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.313158 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.314502 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.314926 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.315030 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:33.319829 (+ 4799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:58957: BlockingRecv error: recv got EOF from 127.18.193.3:58957 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 07 04:10:33 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:33 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:33.329140 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:33.320722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:40259)
0807 04:10:33.320924 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:33.320944 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:33.321057 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:33.321402 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:33.321406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:33.321757 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:33.321905 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:33.321915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.322956 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.322962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:33.323900 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:33.323912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.324661 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.324667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:33.324857 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:33.325491 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:33.325508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:33.328837 (+ 3329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 04:10:33.329322 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:33.320779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:40259 (local address 127.18.193.62:37945)
0807 04:10:33.320947 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:33.320950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:33.321004 (+ 54us) server_negotiation.cc:407] Connection header received
0807 04:10:33.321198 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:33.321203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:33.321273 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:33.321374 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:33.322059 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.322811 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.324045 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.324481 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.324644 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:33.329054 (+ 4410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:40259: BlockingRecv error: recv got EOF from 127.18.193.3:40259 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 04:10:33.329501 28922 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 04:10:33 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:33 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:33.552906 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:33.543235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:48021)
0807 04:10:33.544156 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:33.544176 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:33.544334 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:33.544775 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:33.544779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:33.545309 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:33.545455 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:33.545464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.546535 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.546539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:33.547501 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:33.547514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.548203 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.548208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:33.548326 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:33.548918 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:33.548935 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:33.552489 (+ 3554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":697,"threads_started":1}
W20260807 04:10:33.552956 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:33.543353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:48021 (local address 127.18.193.62:37945)
0807 04:10:33.543623 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:33.543630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:33.544269 (+ 639us) server_negotiation.cc:407] Connection header received
0807 04:10:33.544479 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:33.544485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:33.544613 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:33.544729 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:33.545600 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.546391 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.547637 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:33.548044 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:33.548135 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:33.552743 (+ 4608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:48021: BlockingRecv error: recv got EOF from 127.18.193.2:48021 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 04:10:34 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:34 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:34.009277 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:33.999340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:50545)
0807 04:10:33.999875 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:33.999892 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:34.000012 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:34.000338 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:34.000341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:34.000793 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:34.000982 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:34.000991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.002379 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.002390 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:34.003410 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:34.003426 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.004195 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.004201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:34.004395 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:34.005070 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:34.005087 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:34.008916 (+ 3829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":299,"threads_started":1}
W20260807 04:10:34.009418 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:33.999486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:50545 (local address 127.18.193.62:37945)
0807 04:10:33.999697 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:33.999701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:33.999971 (+ 270us) server_negotiation.cc:407] Connection header received
0807 04:10:34.000130 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:34.000134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:34.000196 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:34.000285 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:34.001148 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.002190 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.003575 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.004026 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.004168 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:34.009156 (+ 4988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:50545: BlockingRecv error: recv got EOF from 127.18.193.1:50545 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 04:10:34 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:34 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:34.340425 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:34.330156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:54571)
0807 04:10:34.330776 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:34.330793 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:34.330912 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:34.331355 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:34.331359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:34.331851 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:34.332012 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:34.332022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.333384 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.333389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:34.334482 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:34.334514 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.335306 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.335311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:34.335437 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:34.336070 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:34.336088 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:34.340180 (+ 4092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":372,"threads_started":1}
W20260807 04:10:34.340703 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:34.330307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:54571 (local address 127.18.193.62:37945)
0807 04:10:34.330495 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:34.330500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:34.330897 (+ 397us) server_negotiation.cc:407] Connection header received
0807 04:10:34.331073 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:34.331077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:34.331158 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:34.331257 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:34.332191 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.333159 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.334662 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.335108 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.335212 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:34.340435 (+ 5223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:54571: BlockingRecv error: recv got EOF from 127.18.193.3:54571 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 04:10:34 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:34 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:34.563611 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:34.554043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:52233)
0807 04:10:34.554609 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:34.554633 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:34.554748 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:34.555063 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:34.555066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:34.555519 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:34.555709 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:34.555720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.556829 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.556833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:34.557822 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:34.557835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.558542 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.558546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:34.558675 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:34.559356 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:34.559375 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:34.563323 (+ 3948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":340,"threads_started":1}
W20260807 04:10:34.563802 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:34.554122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:52233 (local address 127.18.193.62:37945)
0807 04:10:34.554313 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:34.554317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:34.554714 (+ 397us) server_negotiation.cc:407] Connection header received
0807 04:10:34.554855 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:34.554859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:34.554922 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:34.555009 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:34.555860 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.556701 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.557952 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:34.558374 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:34.558468 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:34.563557 (+ 5089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:52233: BlockingRecv error: recv got EOF from 127.18.193.2:52233 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 04:10:35 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:35 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:35.020880 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:35.010551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:51395)
0807 04:10:35.011061 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:35.011078 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:35.011207 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:35.011557 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:35.011560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:35.011919 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:35.012087 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:35.012095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.013341 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.013349 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:35.014353 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:35.014367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.015120 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.015125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:35.015312 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:35.016178 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:35.016202 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:35.020528 (+ 4326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":289,"threads_started":1}
W20260807 04:10:35.021127 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:35.010790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:51395 (local address 127.18.193.62:37945)
0807 04:10:35.011050 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:35.011058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:35.011166 (+ 108us) server_negotiation.cc:407] Connection header received
0807 04:10:35.011339 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:35.011343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:35.011407 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:35.011493 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:35.012266 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.013172 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.014504 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.014971 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.015067 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:35.020846 (+ 5779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:51395: BlockingRecv error: recv got EOF from 127.18.193.1:51395 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 04:10:35 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:35 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:35.351450 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:35.341688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:36109)
0807 04:10:35.342269 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:35.342285 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:35.342405 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:35.342743 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:35.342747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:35.343183 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:35.343369 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:35.343379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.344420 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.344424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:35.345444 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:35.345458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.346374 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.346379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:35.346518 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:35.347237 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:35.347256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:35.351136 (+ 3880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":329,"threads_started":1}
W20260807 04:10:35.351876 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:35.341809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:36109 (local address 127.18.193.62:37945)
0807 04:10:35.342097 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:35.342104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:35.342369 (+ 265us) server_negotiation.cc:407] Connection header received
0807 04:10:35.342525 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:35.342528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:35.342607 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:35.342697 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:35.343527 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.344294 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.345671 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.346193 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.346331 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:35.351387 (+ 5056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:36109: BlockingRecv error: recv got EOF from 127.18.193.3:36109 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 07 04:10:35 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:35 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:35.574364 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:35.564740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:43731)
0807 04:10:35.565361 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:35.565381 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:35.565562 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:35.565918 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:35.565923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:35.566547 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:35.566800 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:35.566810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.567969 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.567977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:35.568911 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:35.568923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.569630 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.569636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:35.569815 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:35.570453 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:35.570474 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:35.574079 (+ 3605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":314,"threads_started":1}
W20260807 04:10:35.574625 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:35.564932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:43731 (local address 127.18.193.62:37945)
0807 04:10:35.565180 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:35.565187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:35.565487 (+ 300us) server_negotiation.cc:407] Connection header received
0807 04:10:35.565702 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:35.565707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:35.565777 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:35.565887 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:35.566981 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.567828 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.569065 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:35.569472 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:35.569615 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:35.574345 (+ 4730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:43731: BlockingRecv error: recv got EOF from 127.18.193.2:43731 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 04:10:36 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:36 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:36.037091 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:36.022198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:36367)
0807 04:10:36.022784 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:36.022801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:36.023013 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:36.023384 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:36.023389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:36.023837 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:36.024021 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:36.024031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.025165 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.025171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:36.026287 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:36.026304 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.027027 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.027033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:36.027229 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:36.032913 (+ 5684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:36.032966 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:36.036702 (+ 3736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":339,"threads_started":1}
W20260807 04:10:36.037221 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:36.022320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:36367 (local address 127.18.193.62:37945)
0807 04:10:36.022605 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:36.022613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:36.022924 (+ 311us) server_negotiation.cc:407] Connection header received
0807 04:10:36.023156 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:36.023163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:36.023235 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:36.023349 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:36.024171 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.025026 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.026445 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.026873 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.027015 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:36.036950 (+ 9935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:36367: BlockingRecv error: recv got EOF from 127.18.193.1:36367 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 07 04:10:36 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:36 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:36.362308 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:36.352714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:43903)
0807 04:10:36.353423 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:36.353439 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:36.353600 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:36.354148 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:36.354153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:36.354549 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:36.354714 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:36.354724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.355806 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.355820 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:36.356874 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:36.356888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.357692 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.357697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:36.357831 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:36.358479 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:36.358496 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:36.362076 (+ 3580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":479,"threads_started":1}
W20260807 04:10:36.362644 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:36.352729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:43903 (local address 127.18.193.62:37945)
0807 04:10:36.353014 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:36.353024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:36.353541 (+ 517us) server_negotiation.cc:407] Connection header received
0807 04:10:36.353784 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:36.353788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:36.353945 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:36.354067 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:36.354890 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.355646 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.357036 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.357502 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.357647 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:36.362331 (+ 4684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:43903: BlockingRecv error: recv got EOF from 127.18.193.3:43903 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 04:10:36 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:36 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:36.585568 28939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:36.575580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:33905)
0807 04:10:36.576382 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:36.576417 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:36.576633 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:36.577063 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:36.577070 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:36.577579 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:36.577827 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:36.577841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.578982 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.578991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:36.580002 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:36.580014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.580763 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.580767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:36.580894 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:36.581527 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:36.581546 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:36.585119 (+ 3573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":503,"threads_started":1}
W20260807 04:10:36.585623 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:36.575761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:33905 (local address 127.18.193.62:37945)
0807 04:10:36.576065 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:36.576076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:36.576516 (+ 440us) server_negotiation.cc:407] Connection header received
0807 04:10:36.576759 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:36.576763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:36.576829 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:36.576967 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:36.578025 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.578805 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.580146 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:36.580610 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:36.580703 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:36.585355 (+ 4652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:33905: BlockingRecv error: recv got EOF from 127.18.193.2:33905 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 04:10:37 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:37 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:37.048306 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:37.038367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:35807)
0807 04:10:37.038883 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:37.038900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:37.039023 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:37.039323 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:37.039326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:37.039765 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:37.039919 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:37.039927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.040952 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.040956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:37.041999 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:37.042013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.042941 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.042946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:37.043086 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:37.043757 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:37.043775 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:37.047692 (+ 3917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":285,"threads_started":1}
W20260807 04:10:37.048427 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:37.038598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:35807 (local address 127.18.193.62:37945)
0807 04:10:37.038828 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:37.038836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:37.039010 (+ 174us) server_negotiation.cc:407] Connection header received
0807 04:10:37.039129 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:37.039133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:37.039197 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:37.039272 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:37.040061 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.040822 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.042187 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.042778 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.042904 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:37.048166 (+ 5262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:35807: BlockingRecv error: recv got EOF from 127.18.193.1:35807 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 04:10:37 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:37 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:37.374200 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:37.363460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:40215)
0807 04:10:37.364016 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:37.364032 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:37.364149 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:37.364509 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:37.364513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:37.364968 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:37.365142 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:37.365151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.366201 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.366205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:37.367225 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:37.367238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.367990 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.367996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:37.368201 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:37.369023 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:37.369048 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:37.373826 (+ 4778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":306,"threads_started":1}
W20260807 04:10:37.374375 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:37.363568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:40215 (local address 127.18.193.62:37945)
0807 04:10:37.363768 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:37.363772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:37.364154 (+ 382us) server_negotiation.cc:407] Connection header received
0807 04:10:37.364242 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:37.364246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:37.364355 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:37.364453 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:37.365310 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.366037 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.367368 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.367825 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.367936 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:37.374075 (+ 6139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:40215: BlockingRecv error: recv got EOF from 127.18.193.3:40215 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 04:10:37 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:37 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:37.596206 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:37.586630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:50143)
0807 04:10:37.587212 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:37.587241 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:37.587361 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:37.587677 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:37.587681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:37.588107 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:37.588253 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:37.588262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.589654 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.589660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:37.590712 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:37.590727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.591361 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.591364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:37.591519 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:37.592108 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:37.592134 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:37.595853 (+ 3719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":326,"threads_started":1}
W20260807 04:10:37.596350 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:37.586807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:50143 (local address 127.18.193.62:37945)
0807 04:10:37.587051 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:37.587056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:37.587346 (+ 290us) server_negotiation.cc:407] Connection header received
0807 04:10:37.587469 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:37.587473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:37.587544 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:37.587635 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:37.588419 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.589468 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.590842 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:37.591239 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:37.591332 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:37.596069 (+ 4737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:50143: BlockingRecv error: recv got EOF from 127.18.193.2:50143 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 04:10:38 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:38 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:38.059260 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:38.049520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:52603)
0807 04:10:38.050336 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:38.050355 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:38.050533 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:38.050915 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:38.050920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:38.051358 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:38.051532 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:38.051541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.052625 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.052632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:38.053637 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:38.053650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.054407 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.054413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:38.054603 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:38.055255 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:38.055272 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:38.059010 (+ 3738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":498,"threads_started":1}
W20260807 04:10:38.059463 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:38.049615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:52603 (local address 127.18.193.62:37945)
0807 04:10:38.049829 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:38.049835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:38.050460 (+ 625us) server_negotiation.cc:407] Connection header received
0807 04:10:38.050675 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:38.050681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:38.050755 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:38.050860 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:38.051687 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.052448 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.053787 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.054241 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.054384 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:38.059217 (+ 4833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:52603: BlockingRecv error: recv got EOF from 127.18.193.1:52603 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 04:10:38 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:38 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:38.387804 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:38.375707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:56485)
0807 04:10:38.376306 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:38.376326 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:38.376490 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:38.377175 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:38.377178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:38.377631 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:38.377823 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:38.377832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.379428 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.379436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:38.380951 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:38.380972 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.381792 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.381798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:38.381972 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:38.382702 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:38.382734 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:38.387469 (+ 4735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":289,"threads_started":1}
W20260807 04:10:38.388005 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:38.375793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:56485 (local address 127.18.193.62:37945)
0807 04:10:38.376020 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:38.376028 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:38.376416 (+ 388us) server_negotiation.cc:407] Connection header received
0807 04:10:38.376685 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:38.376691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:38.376801 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:38.376894 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:38.377968 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.379239 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.381163 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.381633 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.381776 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:38.387735 (+ 5959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:56485: BlockingRecv error: recv got EOF from 127.18.193.3:56485 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 04:10:38 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:38 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:38.607075 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:38.597360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:45153)
0807 04:10:38.597898 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:38.597931 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:38.598103 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:38.598469 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:38.598474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:38.598944 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:38.599119 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:38.599129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.600248 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.600255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:38.601218 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:38.601231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.601968 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.601974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:38.602160 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:38.602813 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:38.602832 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:38.606691 (+ 3859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":296,"threads_started":1}
W20260807 04:10:38.607246 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:38.597467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:45153 (local address 127.18.193.62:37945)
0807 04:10:38.597745 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:38.597754 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:38.598029 (+ 275us) server_negotiation.cc:407] Connection header received
0807 04:10:38.598236 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:38.598242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:38.598315 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:38.598467 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:38.599280 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.600087 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.601374 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:38.601795 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:38.601934 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:38.606952 (+ 5018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:45153: BlockingRecv error: recv got EOF from 127.18.193.2:45153 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 04:10:39 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:39 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:39.071019 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:39.060517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:34873)
0807 04:10:39.061293 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:39.061310 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:39.061427 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:39.061717 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:39.061719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:39.062136 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:39.062326 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:39.062336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.063479 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.063483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:39.065016 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:39.065031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.065849 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.065858 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:39.066038 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:39.066720 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:39.066739 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:39.070757 (+ 4018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":350,"threads_started":1}
W20260807 04:10:39.071139 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:39.060688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:34873 (local address 127.18.193.62:37945)
0807 04:10:39.060906 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:39.060911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:39.061441 (+ 530us) server_negotiation.cc:407] Connection header received
0807 04:10:39.061511 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:39.061514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:39.061578 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:39.061669 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:39.062510 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.063327 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.065162 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.065625 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.065764 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:39.070921 (+ 5157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:34873: BlockingRecv error: recv got EOF from 127.18.193.1:34873 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 04:10:39 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:39 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:39.399124 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:39.389321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:47169)
0807 04:10:39.390043 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:39.390065 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:39.390220 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:39.390631 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:39.390636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:39.391172 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:39.391416 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:39.391427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.392445 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.392449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:39.393508 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:39.393528 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.394236 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.394241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:39.394363 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:39.395018 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:39.395039 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:39.398859 (+ 3820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":455,"threads_started":1}
W20260807 04:10:39.399451 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:39.389462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:47169 (local address 127.18.193.62:37945)
0807 04:10:39.389779 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:39.389788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:39.390180 (+ 392us) server_negotiation.cc:407] Connection header received
0807 04:10:39.390330 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:39.390335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:39.390440 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:39.390565 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:39.391569 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.392301 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.393689 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.394091 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.394230 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:39.399132 (+ 4902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:47169: BlockingRecv error: recv got EOF from 127.18.193.3:47169 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 04:10:39 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:39 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:39.618582 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:39.608248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:49149)
0807 04:10:39.608904 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:39.608927 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:39.609048 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:39.609352 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:39.609355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:39.609801 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:39.609979 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:39.609989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.611115 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.611119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:39.612125 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:39.612139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.613246 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.613254 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:39.613432 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:39.614406 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:39.614424 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:39.618207 (+ 3783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":338,"threads_started":1}
W20260807 04:10:39.618654 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:39.608380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:49149 (local address 127.18.193.62:37945)
0807 04:10:39.608623 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:39.608628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:39.609053 (+ 425us) server_negotiation.cc:407] Connection header received
0807 04:10:39.609142 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:39.609146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:39.609211 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:39.609302 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:39.610149 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.610948 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.612298 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:39.613044 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:39.613199 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:39.618411 (+ 5212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:49149: BlockingRecv error: recv got EOF from 127.18.193.2:49149 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 04:10:40 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:40 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:40.082080 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:40.072148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:59209)
0807 04:10:40.072834 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:40.072852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:40.072969 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:40.073311 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:40.073314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:40.073726 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:40.073891 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:40.073900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.075059 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.075063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:40.076050 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:40.076063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.077056 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.077060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:40.077169 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:40.077825 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:40.077843 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:40.081704 (+ 3861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":282,"threads_started":1}
W20260807 04:10:40.082223 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:40.072204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:59209 (local address 127.18.193.62:37945)
0807 04:10:40.072516 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:40.072523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:40.072949 (+ 426us) server_negotiation.cc:407] Connection header received
0807 04:10:40.073109 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:40.073113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:40.073178 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:40.073266 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:40.074070 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.074928 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.076197 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.076875 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.076978 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:40.081930 (+ 4952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:59209: BlockingRecv error: recv got EOF from 127.18.193.1:59209 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 04:10:40 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:40 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:40.409348 28951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:40.400320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:53095)
0807 04:10:40.400918 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:40.400935 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:40.401065 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:40.401359 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:40.401362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:40.401762 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:40.401934 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:40.401943 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.402940 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.402944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:40.403887 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:40.403899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.404686 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.404690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:40.404802 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:40.405446 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:40.405464 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:40.409086 (+ 3622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":324,"threads_started":1}
W20260807 04:10:40.409490 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:40.400552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:53095 (local address 127.18.193.62:37945)
0807 04:10:40.400806 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:40.400813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:40.401064 (+ 251us) server_negotiation.cc:407] Connection header received
0807 04:10:40.401167 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:40.401171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:40.401234 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:40.401308 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:40.402100 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.402815 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.404017 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.404477 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.404658 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:40.409281 (+ 4623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:53095: BlockingRecv error: recv got EOF from 127.18.193.3:53095 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 04:10:40 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:40 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:40.629668 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:40.619796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:48467)
0807 04:10:40.620353 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:40.620373 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:40.620626 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:40.621100 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:40.621106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:40.621683 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:40.621914 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:40.621926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.623221 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.623225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:40.624189 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:40.624202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.624978 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.624983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:40.625108 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:40.625744 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:40.625767 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:40.629368 (+ 3601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":295,"threads_started":1}
W20260807 04:10:40.629868 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:40.619984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:48467 (local address 127.18.193.62:37945)
0807 04:10:40.620199 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:40.620205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:40.620512 (+ 307us) server_negotiation.cc:407] Connection header received
0807 04:10:40.620767 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:40.620772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:40.620865 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:40.620974 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:40.622073 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.623082 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.624327 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:40.624770 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:40.624885 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:40.629621 (+ 4736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:48467: BlockingRecv error: recv got EOF from 127.18.193.2:48467 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 04:10:41 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:41 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:41.096781 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:41.083440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:33881)
0807 04:10:41.084044 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:41.084065 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:41.084205 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:41.084661 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:41.084665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:41.085091 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:41.085245 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:41.085255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.086296 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.086300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:41.087414 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:41.087428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.088277 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.088283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:41.088484 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:41.092656 (+ 4172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:41.092677 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:41.096452 (+ 3775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":321,"threads_started":1}
W20260807 04:10:41.097160 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:41.083472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:33881 (local address 127.18.193.62:37945)
0807 04:10:41.083712 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:41.083717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:41.084176 (+ 459us) server_negotiation.cc:407] Connection header received
0807 04:10:41.084347 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:41.084351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:41.084416 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:41.084511 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:41.085395 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.086167 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.087553 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.088117 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.088251 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:41.096906 (+ 8655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:33881: BlockingRecv error: recv got EOF from 127.18.193.1:33881 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 04:10:41 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:41 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:41.419633 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:41.410282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:60325)
0807 04:10:41.410820 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:41.410837 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:41.410975 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:41.411268 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:41.411272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:41.411684 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:41.411858 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:41.411867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.413046 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.413051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:41.414018 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:41.414032 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.414778 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.414783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:41.414914 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:41.415613 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:41.415633 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:41.419346 (+ 3713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":296,"threads_started":1}
W20260807 04:10:41.419781 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:41.410510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:60325 (local address 127.18.193.62:37945)
0807 04:10:41.410734 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:41.410740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:41.410970 (+ 230us) server_negotiation.cc:407] Connection header received
0807 04:10:41.411070 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:41.411074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:41.411139 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:41.411227 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:41.412018 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.412875 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.414160 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.414570 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.414674 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:41.419532 (+ 4858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:60325: BlockingRecv error: recv got EOF from 127.18.193.3:60325 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 04:10:41 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:41 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:41.640851 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:41.630812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:36987)
0807 04:10:41.631366 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:41.631412 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:41.631577 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:41.632019 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:41.632023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:41.632427 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:41.632606 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:41.632615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.633756 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.633761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:41.634834 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:41.634848 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.635631 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.635635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:41.635766 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:41.636395 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:41.636415 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:41.640532 (+ 4117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":287,"threads_started":1}
W20260807 04:10:41.641149 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:41.630923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:36987 (local address 127.18.193.62:37945)
0807 04:10:41.631135 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:41.631141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:41.631504 (+ 363us) server_negotiation.cc:407] Connection header received
0807 04:10:41.631755 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:41.631762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:41.631846 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:41.631963 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:41.632778 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.633610 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.635009 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:41.635449 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:41.635564 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:41.640841 (+ 5277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:36987: BlockingRecv error: recv got EOF from 127.18.193.2:36987 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 04:10:42 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:42 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:42.107848 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:42.097942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:53445)
0807 04:10:42.098533 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:42.098549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:42.098712 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:42.099068 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:42.099073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:42.099533 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:42.099705 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:42.099715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.100957 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.100963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:42.102262 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:42.102275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.102995 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.103001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:42.103187 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:42.103812 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:42.103831 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:42.107536 (+ 3705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":320,"threads_started":1}
W20260807 04:10:42.108028 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:42.098041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:53445 (local address 127.18.193.62:37945)
0807 04:10:42.098333 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:42.098339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:42.098640 (+ 301us) server_negotiation.cc:407] Connection header received
0807 04:10:42.098847 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:42.098852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:42.098924 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:42.099032 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:42.099861 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.100800 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.102423 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.102833 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.102971 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:42.107743 (+ 4772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:53445: BlockingRecv error: recv got EOF from 127.18.193.1:53445 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 07 04:10:42 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:42 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:42.430632 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:42.420755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:49977)
0807 04:10:42.421307 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:42.421324 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:42.421450 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:42.421787 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:42.421790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:42.422192 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:42.422344 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:42.422353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.423477 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.423482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:42.424484 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:42.424499 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.425445 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.425449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:42.425615 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:42.426245 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:42.426263 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:42.430330 (+ 4067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":302,"threads_started":1}
W20260807 04:10:42.430919 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:42.420830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:49977 (local address 127.18.193.62:37945)
0807 04:10:42.421139 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:42.421146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:42.421437 (+ 291us) server_negotiation.cc:407] Connection header received
0807 04:10:42.421567 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:42.421572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:42.421637 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:42.421728 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:42.422497 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.423268 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.424818 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.425268 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.425379 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:42.430606 (+ 5227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:49977: BlockingRecv error: recv got EOF from 127.18.193.3:49977 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 04:10:42 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:42 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:42.651557 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:42.642100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:46257)
0807 04:10:42.642685 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:42.642707 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:42.642886 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:42.643286 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:42.643291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:42.643771 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:42.643942 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:42.643957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.645181 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.645185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:42.646339 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:42.646353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.647130 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.647135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:42.647259 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:42.647902 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:42.647921 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:42.651323 (+ 3402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":302,"threads_started":1}
W20260807 04:10:42.651870 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:42.642297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:46257 (local address 127.18.193.62:37945)
0807 04:10:42.642558 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:42.642564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:42.642810 (+ 246us) server_negotiation.cc:407] Connection header received
0807 04:10:42.643027 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:42.643040 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:42.643107 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:42.643240 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:42.644102 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.644997 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.646527 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:42.646971 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:42.647073 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:42.651557 (+ 4484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:46257: BlockingRecv error: recv got EOF from 127.18.193.2:46257 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 04:10:43 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:43 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:43.118718 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:43.108989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:38653)
0807 04:10:43.109537 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:43.109556 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:43.109718 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:43.110092 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:43.110097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:43.110538 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:43.110723 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:43.110732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.111852 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.111857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:43.112894 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:43.112907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.113619 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.113626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:43.113810 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:43.114478 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:43.114496 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:43.118358 (+ 3862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":282,"threads_started":1}
W20260807 04:10:43.118857 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:43.109184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:38653 (local address 127.18.193.62:37945)
0807 04:10:43.109461 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:43.109467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:43.109641 (+ 174us) server_negotiation.cc:407] Connection header received
0807 04:10:43.109856 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:43.109861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:43.109935 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:43.110045 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:43.110871 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.111724 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.113035 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.113452 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.113602 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:43.118596 (+ 4994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:38653: BlockingRecv error: recv got EOF from 127.18.193.1:38653 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Aug 07 04:10:43 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:43 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:43.442731 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:43.432239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:37125)
0807 04:10:43.432893 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:43.432910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:43.433047 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:43.433569 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:43.433574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:43.434103 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:43.434317 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:43.434328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.435663 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.435670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:43.436697 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:43.436710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.437491 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.437498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:43.437675 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:43.438357 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:43.438375 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:43.442354 (+ 3979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":404,"threads_started":1}
W20260807 04:10:43.442904 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:43.432312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:37125 (local address 127.18.193.62:37945)
0807 04:10:43.432611 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:43.432621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:43.433020 (+ 399us) server_negotiation.cc:407] Connection header received
0807 04:10:43.433280 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:43.433285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:43.433373 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:43.433530 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:43.434477 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.435528 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.436845 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.437302 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.437447 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:43.442602 (+ 5155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:37125: BlockingRecv error: recv got EOF from 127.18.193.3:37125 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 04:10:43 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:43 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:43.663329 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:43.652893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:33703)
0807 04:10:43.653610 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:43.653632 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:43.653794 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:43.654151 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:43.654156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:43.654615 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:43.654797 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:43.654806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.655865 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.655871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:43.657214 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:43.657230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.658069 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.658075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:43.658255 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:43.659061 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:43.659085 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:43.663081 (+ 3996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":472,"threads_started":1}
W20260807 04:10:43.663640 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:43.653048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:33703 (local address 127.18.193.62:37945)
0807 04:10:43.653481 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:43.653487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:43.653718 (+ 231us) server_negotiation.cc:407] Connection header received
0807 04:10:43.653926 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:43.653933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:43.654002 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:43.654112 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:43.654950 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.655717 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.657390 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:43.657885 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:43.658037 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:43.663330 (+ 5293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:33703: BlockingRecv error: recv got EOF from 127.18.193.2:33703 (error 108)
Metrics: {"server-negotiator.queue_time_us":216}
Aug 07 04:10:44 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:44 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:44.129223 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:44.119832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:46415)
0807 04:10:44.120399 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:44.120415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:44.120618 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:44.120990 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:44.120997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:44.121422 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:44.121630 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:44.121640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.122690 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.122696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:44.123670 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:44.123681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.124431 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.124437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:44.124644 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:44.125372 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:44.125389 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:44.128875 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":311,"threads_started":1}
W20260807 04:10:44.129374 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:44.119904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:46415 (local address 127.18.193.62:37945)
0807 04:10:44.120135 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:44.120143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:44.120504 (+ 361us) server_negotiation.cc:407] Connection header received
0807 04:10:44.120756 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:44.120762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:44.120839 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:44.120944 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:44.121785 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.122546 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.123815 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.124247 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.124384 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:44.129098 (+ 4714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:46415: BlockingRecv error: recv got EOF from 127.18.193.1:46415 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 04:10:44 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:44 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:44.454298 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:44.443864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:44357)
0807 04:10:44.444458 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:44.444473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:44.444616 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:44.444930 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:44.444933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:44.445365 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:44.445546 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:44.445555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.446659 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.446664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:44.447810 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:44.447824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.449089 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.449093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:44.449223 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:44.449849 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:44.449866 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:44.454044 (+ 4178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":330,"threads_started":1}
W20260807 04:10:44.454620 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:44.443978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:44357 (local address 127.18.193.62:37945)
0807 04:10:44.444161 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:44.444173 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:44.444672 (+ 499us) server_negotiation.cc:407] Connection header received
0807 04:10:44.444735 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:44.444739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:44.444802 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:44.444887 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:44.445738 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.446502 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.448125 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.448929 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.449036 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:44.454319 (+ 5283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:44357: BlockingRecv error: recv got EOF from 127.18.193.3:44357 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 04:10:44 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:44 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:44.675244 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:44.664426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:46005)
0807 04:10:44.665060 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:44.665082 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:44.665269 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:44.665739 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:44.665745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:44.666369 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:44.666645 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:44.666658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.668035 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.668041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:44.669143 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:44.669157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.670036 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.670044 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:44.670240 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:44.670912 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:44.670930 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:44.674946 (+ 4016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":362,"threads_started":1}
W20260807 04:10:44.675590 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:44.664525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:46005 (local address 127.18.193.62:37945)
0807 04:10:44.664870 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:44.664879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:44.665219 (+ 340us) server_negotiation.cc:407] Connection header received
0807 04:10:44.665450 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:44.665458 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:44.665545 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:44.665682 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:44.666874 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.667848 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.669316 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:44.669814 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:44.669968 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:44.675259 (+ 5291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:46005: BlockingRecv error: recv got EOF from 127.18.193.2:46005 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 07 04:10:45 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:45 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:45.140308 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:45.130527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:59713)
0807 04:10:45.131076 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:45.131092 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:45.131209 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:45.131659 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:45.131681 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:45.132140 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:45.132293 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:45.132302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.133681 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.133687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:45.134717 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:45.134732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.135471 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.135476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:45.135610 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:45.136249 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:45.136266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:45.140017 (+ 3751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":312,"threads_started":1}
W20260807 04:10:45.140681 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:45.130540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:59713 (local address 127.18.193.62:37945)
0807 04:10:45.130800 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:45.130805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:45.131190 (+ 385us) server_negotiation.cc:407] Connection header received
0807 04:10:45.131314 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:45.131318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:45.131394 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:45.131619 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:45.132470 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.133471 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.134885 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.135332 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.135435 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:45.140236 (+ 4801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:59713: BlockingRecv error: recv got EOF from 127.18.193.1:59713 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 07 04:10:45 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:45 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:45.465446 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:45.455572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:37473)
0807 04:10:45.456239 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:45.456255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:45.456421 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:45.456883 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:45.456888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:45.457301 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:45.457473 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:45.457482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.458598 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.458604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:45.459795 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:45.459811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.460673 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.460680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:45.460920 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:45.461515 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:45.461534 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:45.465086 (+ 3552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":348,"threads_started":1}
W20260807 04:10:45.465651 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:45.455658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:37473 (local address 127.18.193.62:37945)
0807 04:10:45.455983 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:45.456001 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:45.456346 (+ 345us) server_negotiation.cc:407] Connection header received
0807 04:10:45.456625 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:45.456631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:45.456717 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:45.456832 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:45.457620 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.458446 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.459947 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.460455 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.460626 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:45.465298 (+ 4672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:37473: BlockingRecv error: recv got EOF from 127.18.193.3:37473 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 07 04:10:45 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:45 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:45.685643 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:45.676229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:34629)
0807 04:10:45.676776 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:45.676797 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:45.676960 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:45.677307 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:45.677311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:45.677765 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:45.677923 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:45.677933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.678933 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.678939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:45.679882 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:45.679894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.680878 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.680884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:45.681064 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:45.681736 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:45.681754 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:45.685391 (+ 3637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":340,"threads_started":1}
W20260807 04:10:45.685897 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:45.676435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:34629 (local address 127.18.193.62:37945)
0807 04:10:45.676708 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:45.676714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:45.676880 (+ 166us) server_negotiation.cc:407] Connection header received
0807 04:10:45.677086 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:45.677094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:45.677166 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:45.677269 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:45.678068 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.678792 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.680037 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:45.680722 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:45.680854 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:45.685625 (+ 4771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:34629: BlockingRecv error: recv got EOF from 127.18.193.2:34629 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 04:10:46 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:46 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:46.156757 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:46.141608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:50251)
0807 04:10:46.142148 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:46.142164 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:46.142333 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:46.142665 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:46.142668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:46.143043 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:46.143242 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:46.143251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.144389 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.144392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:46.145383 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:46.145395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.146151 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.146157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:46.146334 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:46.152232 (+ 5898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:46.152251 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:46.156322 (+ 4071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":241,"threads_started":1}
W20260807 04:10:46.156898 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:46.141834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:50251 (local address 127.18.193.62:37945)
0807 04:10:46.142065 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:46.142072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:46.142284 (+ 212us) server_negotiation.cc:407] Connection header received
0807 04:10:46.142443 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:46.142447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:46.142516 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:46.142600 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:46.143409 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.144251 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.145536 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.145987 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.146127 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:46.156623 (+ 10496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:50251: BlockingRecv error: recv got EOF from 127.18.193.1:50251 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 04:10:46 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:46 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.3@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:46.476764 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:46.466522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.3:43661)
0807 04:10:46.467018 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:46.467036 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:46.467168 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:46.467485 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:46.467488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:46.467921 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:46.468073 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:46.468082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.469415 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.469425 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:46.470729 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:46.470744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.471624 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.471630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:46.471823 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:46.472466 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:46.472485 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:46.476377 (+ 3892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":245,"threads_started":1}
W20260807 04:10:46.476902 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:46.466638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:43661 (local address 127.18.193.62:37945)
0807 04:10:46.466982 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:46.466991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:46.467137 (+ 146us) server_negotiation.cc:407] Connection header received
0807 04:10:46.467282 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:46.467286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:46.467351 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:46.467439 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:46.468251 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.469234 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.470889 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.471454 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.471639 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:46.476646 (+ 5007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.3:43661: BlockingRecv error: recv got EOF from 127.18.193.3:43661 (error 108)
Metrics: {"server-negotiator.queue_time_us":191}
Aug 07 04:10:46 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:46 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.2@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:46.697535 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:46.687285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.2:43951)
0807 04:10:46.688034 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:46.688069 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:46.688220 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:46.688761 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:46.688766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:46.689206 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:46.689427 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:46.689438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.690623 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.690627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:46.691672 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:46.691685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.692620 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.692628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:46.692787 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:46.693491 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:46.693508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:46.697160 (+ 3652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":373,"threads_started":1}
W20260807 04:10:46.697820 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:46.687384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:43951 (local address 127.18.193.62:37945)
0807 04:10:46.688181 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:46.688190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:46.688231 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:10:46.688379 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:46.688384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:46.688469 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:46.688716 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:46.689632 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.690457 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.691838 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:46.692358 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:46.692528 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:46.697449 (+ 4921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.2:43951: BlockingRecv error: recv got EOF from 127.18.193.2:43951 (error 108)
Metrics: {"server-negotiator.queue_time_us":189}
Aug 07 04:10:47 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:10:47 dist-test-slave-8n49 krb5kdc[27873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.193.1@KRBTEST.COM for oryx/127.18.193.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:10:47.168371 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:47.158160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37945 (local address 127.18.193.1:48047)
0807 04:10:47.158888 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:47.158908 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:47.159057 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:47.159424 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:47.159429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:47.160082 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:47.160386 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:47.160399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.161676 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.161681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:47.162648 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:47.162660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.163444 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.163449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:47.163578 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:47.164291 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:47.164309 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:47.168129 (+ 3820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.193.62:37945: Server oryx/127.18.193.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":444,"threads_started":1}
W20260807 04:10:47.168567 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:47.158260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:48047 (local address 127.18.193.62:37945)
0807 04:10:47.158463 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:47.158470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:47.159029 (+ 559us) server_negotiation.cc:407] Connection header received
0807 04:10:47.159202 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:47.159206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:47.159272 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:47.159359 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:47.160752 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.161509 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.162792 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.163239 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.163338 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:47.168342 (+ 5004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.193.1:48047: BlockingRecv error: recv got EOF from 127.18.193.1:48047 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:10:47.182753 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28492
I20260807 04:10:47.197166 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:47.184043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.18.193.1:37299)
0807 04:10:47.184919 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:47.184928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:47.184963 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:10:47.185058 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:47.185066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:47.185338 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:47.185530 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:47.187085 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.188319 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.189674 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.190272 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.190390 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:47.193704 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:47.193713 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:47.193728 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:47.193771 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:47.196625 (+ 2854us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.193.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 04:10:47.196922 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48102: Request ticket server kudu/127.18.193.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":431,"threads_started":1}
W20260807 04:10:47.197300 28975 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48102: Request ticket server kudu/127.18.193.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 04:10:47.197363 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28492: Not authorized: Client connection negotiation failed: client connection to 127.18.193.1:37299: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.193.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 04:10:47.197665 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 28492
I20260807 04:10:47.219548 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28650
I20260807 04:10:47.234665 28976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:47.220854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55424 (local address 127.18.193.2:43019)
0807 04:10:47.221451 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:47.221460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:47.221487 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:10:47.221564 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:47.221574 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:47.221830 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:47.222093 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:47.223472 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.224645 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.226037 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.226830 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.227001 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:47.230513 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:47.230527 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:47.230547 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:47.230601 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:47.234076 (+ 3475us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.193.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 04:10:47.234377 (+ 301us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55424: Request ticket server kudu/127.18.193.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":305,"threads_started":1}
W20260807 04:10:47.234783 28976 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55424: Request ticket server kudu/127.18.193.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 04:10:47.234944 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28650: Not authorized: Client connection negotiation failed: client connection to 127.18.193.2:43019: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.193.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 04:10:47.235102 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 28650
I20260807 04:10:47.254624 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28788
I20260807 04:10:47.269191 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:47.255897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.18.193.3:40157)
0807 04:10:47.256490 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:47.256499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:47.256524 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:10:47.256690 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:47.256698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:47.257031 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:47.257254 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:47.259200 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.260441 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.261801 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.262483 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.262600 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:47.265285 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:47.265294 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:47.265308 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:47.265347 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:47.268367 (+ 3020us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.193.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 04:10:47.268796 (+ 429us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48046: Request ticket server kudu/127.18.193.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":305,"threads_started":1}
W20260807 04:10:47.269207 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28788: Not authorized: Client connection negotiation failed: client connection to 127.18.193.3:40157: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.193.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 04:10:47.269309 28977 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48046: Request ticket server kudu/127.18.193.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 04:10:47.269387 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 28788
I20260807 04:10:47.287532 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28418
I20260807 04:10:47.288990 28485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:47.412997 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 28418
2026-08-07T04:10:47Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22089 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T04:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:47Z Disabled control of system clock
I20260807 04:10:47.502123 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38691
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:38691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 04:10:47.848460 28982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:47.848805 28982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:47.848898 28982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:47.856371 28982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:47.856490 28982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:47.856528 28982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:47.856669 28982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:47.856724 28982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:47.866182 28982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:38691
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:47.867479 28982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:47.869361 28982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:47.882592 28987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:47.882617 28988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:47.882893 28982 server_base.cc:1034] running on GCE node
W20260807 04:10:47.883124 28990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:47.884107 28982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:47.885717 28982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:47.886998 28982 hybrid_clock.cc:648] HybridClock initialized: now 1786075847886930 us; error 53 us; skew 500 ppm
I20260807 04:10:47.887415 28982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:47.889899 28982 webserver.cc:460] Webserver started at http://127.18.193.62:45053/ using document root <none> and password file <none>
I20260807 04:10:47.890650 28982 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:47.890790 28982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:47.891160 28982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:47.894292 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "820c35b799e84cf79a0a6f115db3c86b"
format_stamp: "Formatted at 2026-08-07 04:10:47 on dist-test-slave-8n49"
server_key: "7482b6aba0ec65cfcfe1f4f59b5c1ea4"
server_key_iv: "095c5cbe43bb1dfa7d4267769d3a2f3d"
server_key_version: "encryptionkey@0"
I20260807 04:10:47.895162 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "820c35b799e84cf79a0a6f115db3c86b"
format_stamp: "Formatted at 2026-08-07 04:10:47 on dist-test-slave-8n49"
server_key: "7482b6aba0ec65cfcfe1f4f59b5c1ea4"
server_key_iv: "095c5cbe43bb1dfa7d4267769d3a2f3d"
server_key_version: "encryptionkey@0"
I20260807 04:10:47.901065 28982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 04:10:47.905081 28996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:47.906639 28982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:10:47.906858 28982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "820c35b799e84cf79a0a6f115db3c86b"
format_stamp: "Formatted at 2026-08-07 04:10:47 on dist-test-slave-8n49"
server_key: "7482b6aba0ec65cfcfe1f4f59b5c1ea4"
server_key_iv: "095c5cbe43bb1dfa7d4267769d3a2f3d"
server_key_version: "encryptionkey@0"
I20260807 04:10:47.907128 28982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:47.960317 28982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:47.961261 28982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:47.961582 28982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:47.981436 28982 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:38691
I20260807 04:10:47.981485 29052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:38691 every 8 connection(s)
I20260807 04:10:47.982939 28982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:47.987107 29053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:47.989130 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 28982
I20260807 04:10:47.989358 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:47.990232 19204 external_mini_cluster.cc:1428] Setting key 5ea89c818ac64fe5e5cbdedfb176348e
I20260807 04:10:47.999671 29053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b: Bootstrap starting.
I20260807 04:10:48.003022 29053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:48.004181 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:47.992503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.18.193.62:38691)
0807 04:10:47.993610 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:47.993621 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:47.993664 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:10:47.995381 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:47.995403 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:47.995863 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:47.996256 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:47.996996 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:47.998273 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:47.999603 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:48.000363 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:48.000599 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:48.001827 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:48.001834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:48.001848 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:48.002381 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:48.002455 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:48.002476 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:48.002606 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:48.002832 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 04:10:48.003237 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":373,"threads_started":1}
I20260807 04:10:48.004204 29053 log.cc:824] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:48.007364 29053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b: No bootstrap required, opened a new log
I20260807 04:10:48.015578 29053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820c35b799e84cf79a0a6f115db3c86b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38691 } }
I20260807 04:10:48.015982 29053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:48.016098 29053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820c35b799e84cf79a0a6f115db3c86b, State: Initialized, Role: FOLLOWER
I20260807 04:10:48.016768 29053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820c35b799e84cf79a0a6f115db3c86b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38691 } }
I20260807 04:10:48.017014 29053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:48.017149 29053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:48.017335 29053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:48.019898 29053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820c35b799e84cf79a0a6f115db3c86b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38691 } }
I20260807 04:10:48.020514 29053 leader_election.cc:302] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 820c35b799e84cf79a0a6f115db3c86b; no voters:
I20260807 04:10:48.021229 29053 leader_election.cc:288] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:48.021397 29058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:48.022150 29058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [term 1 LEADER]: Becoming Leader. State: Replica: 820c35b799e84cf79a0a6f115db3c86b, State: Running, Role: LEADER
I20260807 04:10:48.022770 29058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820c35b799e84cf79a0a6f115db3c86b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38691 } }
I20260807 04:10:48.023415 29053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:48.025933 29059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "820c35b799e84cf79a0a6f115db3c86b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820c35b799e84cf79a0a6f115db3c86b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38691 } } }
I20260807 04:10:48.026337 29059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:48.026104 29060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 820c35b799e84cf79a0a6f115db3c86b. Latest consensus state: current_term: 1 leader_uuid: "820c35b799e84cf79a0a6f115db3c86b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820c35b799e84cf79a0a6f115db3c86b" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38691 } } }
I20260807 04:10:48.026547 29060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:48.029410 29068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:48.034660 29068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:48.044184 29068 catalog_manager.cc:1347] Generated new cluster ID: 65e811b8988e4be3a2b31c28a743f8f5
I20260807 04:10:48.044319 29068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:48.068082 29068 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:48.070024 29068 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:48.088368 29068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 820c35b799e84cf79a0a6f115db3c86b: Generated new TSK 0
I20260807 04:10:48.089537 29068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:10:48.108011 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38691
--builtin_ntp_servers=127.18.193.20:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 04:10:48.456490 29079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:48.456890 29079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:48.457059 29079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:48.464898 29079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:48.465039 29079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:48.465211 29079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:48.474864 29079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:48.476260 29079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:48.478183 29079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:48.491874 29084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:48.491819 29087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:48.492297 29085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:48.492868 29079 server_base.cc:1034] running on GCE node
I20260807 04:10:48.493520 29079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:48.494525 29079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:48.495754 29079 hybrid_clock.cc:648] HybridClock initialized: now 1786075848495706 us; error 43 us; skew 500 ppm
I20260807 04:10:48.496089 29079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:48.498448 29079 webserver.cc:460] Webserver started at http://127.18.193.1:35605/ using document root <none> and password file <none>
I20260807 04:10:48.499179 29079 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:48.499315 29079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:48.499657 29079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:48.502718 29079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "d01292ed3b3740d284fcebda8ae4afc9"
format_stamp: "Formatted at 2026-08-07 04:10:48 on dist-test-slave-8n49"
server_key: "1b77439ea901a3295e1c659aad9cc063"
server_key_iv: "56765d48b6049434a5b3523d052cc7d7"
server_key_version: "encryptionkey@0"
I20260807 04:10:48.503575 29079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "d01292ed3b3740d284fcebda8ae4afc9"
format_stamp: "Formatted at 2026-08-07 04:10:48 on dist-test-slave-8n49"
server_key: "1b77439ea901a3295e1c659aad9cc063"
server_key_iv: "56765d48b6049434a5b3523d052cc7d7"
server_key_version: "encryptionkey@0"
I20260807 04:10:48.509166 29079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 04:10:48.512888 29093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:48.514384 29079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:10:48.514597 29079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "d01292ed3b3740d284fcebda8ae4afc9"
format_stamp: "Formatted at 2026-08-07 04:10:48 on dist-test-slave-8n49"
server_key: "1b77439ea901a3295e1c659aad9cc063"
server_key_iv: "56765d48b6049434a5b3523d052cc7d7"
server_key_version: "encryptionkey@0"
I20260807 04:10:48.514847 29079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:48.535255 29079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:48.536178 29079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:48.536520 29079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:48.537616 29079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:48.539149 29079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:48.539280 29079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:48.539422 29079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:48.539513 29079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:48.574760 29079 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:35139
I20260807 04:10:48.574815 29211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:35139 every 8 connection(s)
I20260807 04:10:48.576193 29079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:48.577016 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 29079
I20260807 04:10:48.577270 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:10:48.578094 19204 external_mini_cluster.cc:1428] Setting key 315d69b4832b890374364fb087b6ea49
I20260807 04:10:48.581913 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38691
--builtin_ntp_servers=127.18.193.20:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 04:10:48.593084 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:48.579306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:46871 (local address 127.18.193.62:38691)
0807 04:10:48.579832 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:48.579839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:48.581324 (+ 1485us) server_negotiation.cc:407] Connection header received
0807 04:10:48.584410 (+ 3086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:48.584415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:48.584485 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:48.584673 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:48.586144 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:48.586924 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:48.589672 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:48.590107 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:48.590218 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:48.592011 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:48.592016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:48.592019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:48.592061 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:48.592102 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:48.592114 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:48.592188 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:48.592477 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 04:10:48.592615 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":278,"threads_started":1}
I20260807 04:10:48.593885 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:48.579957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38691 (local address 127.18.193.1:46871)
0807 04:10:48.581151 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:48.581202 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:48.584079 (+ 2877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:48.584926 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:48.584945 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:48.585084 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:48.585941 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:48.585969 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:48.587106 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:48.587114 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:48.589497 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:48.589516 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:48.590368 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:48.590378 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:48.590736 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:48.591608 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:10:48.591614 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:48.591673 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:10:48.591795 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:48.592222 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:48.592236 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:48.592247 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 04:10:48.592466 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 04:10:48.592891 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":337,"threads_started":1}
I20260807 04:10:48.595710 29212 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38691
I20260807 04:10:48.596051 29212 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:48.596917 29212 heartbeater.cc:511] Master 127.18.193.62:38691 requested a full tablet report, sending...
I20260807 04:10:48.599306 29015 ts_manager.cc:194] Registered new tserver with Master: d01292ed3b3740d284fcebda8ae4afc9 (127.18.193.1:35139)
I20260807 04:10:48.601676 29015 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.1:46871
W20260807 04:10:48.929060 29216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:48.929375 29216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:48.929528 29216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:48.937289 29216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:48.937408 29216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:48.937594 29216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:48.947120 29216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:48.948518 29216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:48.950412 29216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:48.965186 29225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:48.965252 29222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:48.965565 29221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:48.967289 29216 server_base.cc:1034] running on GCE node
I20260807 04:10:48.968364 29216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:48.969666 29216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:48.970947 29216 hybrid_clock.cc:648] HybridClock initialized: now 1786075848970826 us; error 118 us; skew 500 ppm
I20260807 04:10:48.971376 29216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:48.974587 29216 webserver.cc:460] Webserver started at http://127.18.193.2:44667/ using document root <none> and password file <none>
I20260807 04:10:48.975621 29216 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:48.975771 29216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:48.976183 29216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:48.981021 29216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "8d4c13ce87c94d408ae3d92fae4233ea"
format_stamp: "Formatted at 2026-08-07 04:10:48 on dist-test-slave-8n49"
server_key: "dc28efc43917a541ec75a787ba308734"
server_key_iv: "8e5854ee5dbd76e92d7482dcb2f5e5ba"
server_key_version: "encryptionkey@0"
I20260807 04:10:48.982347 29216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "8d4c13ce87c94d408ae3d92fae4233ea"
format_stamp: "Formatted at 2026-08-07 04:10:48 on dist-test-slave-8n49"
server_key: "dc28efc43917a541ec75a787ba308734"
server_key_iv: "8e5854ee5dbd76e92d7482dcb2f5e5ba"
server_key_version: "encryptionkey@0"
I20260807 04:10:48.988634 29216 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 04:10:48.992502 29232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:48.994093 29216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:10:48.994304 29216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "8d4c13ce87c94d408ae3d92fae4233ea"
format_stamp: "Formatted at 2026-08-07 04:10:48 on dist-test-slave-8n49"
server_key: "dc28efc43917a541ec75a787ba308734"
server_key_iv: "8e5854ee5dbd76e92d7482dcb2f5e5ba"
server_key_version: "encryptionkey@0"
I20260807 04:10:48.994542 29216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:49.006453 29216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:49.007373 29216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:49.007699 29216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:49.008816 29216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:49.010327 29216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:49.010453 29216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:49.010607 29216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:49.010710 29216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:49.048336 29216 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:46795
I20260807 04:10:49.048388 29354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:46795 every 8 connection(s)
I20260807 04:10:49.049784 29216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:49.054473 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 29216
I20260807 04:10:49.054678 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:10:49.055122 19204 external_mini_cluster.cc:1428] Setting key f602c5ee133d8f6bc65f8dad901aad1e
I20260807 04:10:49.058104 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38691
--builtin_ntp_servers=127.18.193.20:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 04:10:49.063434 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:49.052453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:47377 (local address 127.18.193.62:38691)
0807 04:10:49.052675 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:49.052680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:49.054170 (+ 1490us) server_negotiation.cc:407] Connection header received
0807 04:10:49.056236 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:49.056242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:49.056336 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:49.056450 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:49.057709 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.058519 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.060095 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.060489 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.060704 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:49.062464 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:49.062476 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:49.062483 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:49.062510 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:49.062551 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:49.062568 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:49.062629 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:49.062952 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 04:10:49.063065 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:10:49.065409 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:49.053020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38691 (local address 127.18.193.2:47377)
0807 04:10:49.053970 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:49.054013 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:49.055959 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:49.056713 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:49.056721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:49.056815 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:49.057530 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:49.057551 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.058668 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.058680 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:49.059928 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:49.059941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.060795 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.060804 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:49.061044 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:49.062034 (+ 990us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:10:49.062040 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:49.062098 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:10:49.062223 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:49.062711 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:49.062723 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:49.062733 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:10:49.063639 (+ 906us) client_negotiation.cc:240] Negotiation successful
0807 04:10:49.064074 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":313,"threads_started":1}
I20260807 04:10:49.068935 29355 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38691
I20260807 04:10:49.069276 29355 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:49.070075 29355 heartbeater.cc:511] Master 127.18.193.62:38691 requested a full tablet report, sending...
I20260807 04:10:49.071697 29015 ts_manager.cc:194] Registered new tserver with Master: 8d4c13ce87c94d408ae3d92fae4233ea (127.18.193.2:46795)
I20260807 04:10:49.072978 29015 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.2:47377
W20260807 04:10:49.417184 29358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:49.417495 29358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:49.417579 29358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:49.425628 29358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:49.425750 29358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:49.425858 29358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:49.436131 29358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:49.437611 29358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:49.439435 29358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:49.454545 29363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:49.454625 29366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:49.454610 29364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:49.455596 29358 server_base.cc:1034] running on GCE node
I20260807 04:10:49.456279 29358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:49.457430 29358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:49.458683 29358 hybrid_clock.cc:648] HybridClock initialized: now 1786075849458616 us; error 50 us; skew 500 ppm
I20260807 04:10:49.459153 29358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:49.461987 29358 webserver.cc:460] Webserver started at http://127.18.193.3:45309/ using document root <none> and password file <none>
I20260807 04:10:49.462929 29358 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:49.463037 29358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:49.463444 29358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:49.466993 29358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "3aa6ce04d2a247caba83fb3127f8ad84"
format_stamp: "Formatted at 2026-08-07 04:10:49 on dist-test-slave-8n49"
server_key: "57b247548664f5825fcbd110a32d1284"
server_key_iv: "33f17cc712b411bc61a41e547f1a67dd"
server_key_version: "encryptionkey@0"
I20260807 04:10:49.468171 29358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "3aa6ce04d2a247caba83fb3127f8ad84"
format_stamp: "Formatted at 2026-08-07 04:10:49 on dist-test-slave-8n49"
server_key: "57b247548664f5825fcbd110a32d1284"
server_key_iv: "33f17cc712b411bc61a41e547f1a67dd"
server_key_version: "encryptionkey@0"
I20260807 04:10:49.474936 29358 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 04:10:49.478824 29372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:49.480396 29358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:10:49.480630 29358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "3aa6ce04d2a247caba83fb3127f8ad84"
format_stamp: "Formatted at 2026-08-07 04:10:49 on dist-test-slave-8n49"
server_key: "57b247548664f5825fcbd110a32d1284"
server_key_iv: "33f17cc712b411bc61a41e547f1a67dd"
server_key_version: "encryptionkey@0"
I20260807 04:10:49.480854 29358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:49.496150 29358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:49.497107 29358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:49.497418 29358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:49.498513 29358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:49.500234 29358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:49.500322 29358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:49.500479 29358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:49.500619 29358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:49.538280 29358 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:43471
I20260807 04:10:49.538342 29490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:43471 every 8 connection(s)
I20260807 04:10:49.539722 29358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:49.549418 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 29358
I20260807 04:10:49.549619 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:10:49.550190 19204 external_mini_cluster.cc:1428] Setting key 7d986d7eac4edfa875e1fb3a890738ae
I20260807 04:10:49.555624 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:49.542693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:56015 (local address 127.18.193.62:38691)
0807 04:10:49.542918 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:49.542925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:49.544948 (+ 2023us) server_negotiation.cc:407] Connection header received
0807 04:10:49.547370 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:49.547377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:49.547473 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:49.547579 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:49.549104 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.550272 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.552485 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.553149 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.553287 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:49.554883 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:49.554888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:49.554891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:49.554910 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:49.554948 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:49.554961 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:49.555045 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:49.555279 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 04:10:49.555363 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 04:10:49.556766 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:49.543155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38691 (local address 127.18.193.3:56015)
0807 04:10:49.544614 (+ 1459us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:49.544684 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:49.546981 (+ 2297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:49.547876 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:49.547895 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:49.548012 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:49.548860 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:49.548884 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.550473 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.550477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:49.552305 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:49.552320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.553397 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.553403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:49.553743 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:49.554547 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:10:49.554549 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:49.554595 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:10:49.554683 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:49.555056 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:49.555064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:49.555071 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:10:49.555229 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 04:10:49.555566 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":310,"threads_started":1}
I20260807 04:10:49.558329 29491 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38691
I20260807 04:10:49.558667 29491 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:49.559484 29491 heartbeater.cc:511] Master 127.18.193.62:38691 requested a full tablet report, sending...
I20260807 04:10:49.561125 29015 ts_manager.cc:194] Registered new tserver with Master: 3aa6ce04d2a247caba83fb3127f8ad84 (127.18.193.3:43471)
I20260807 04:10:49.562032 29015 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.3:56015
I20260807 04:10:49.566748 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 04:10:49.574179 29214 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:49.571953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.18.193.62:38691)
0807 04:10:49.572259 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:49.572271 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:49.573013 (+ 742us) server_negotiation.cc:407] Connection header received
0807 04:10:49.573198 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:49.573203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:49.573292 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:49.573384 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:49.573868 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43190: BlockingRecv error: recv got EOF from 127.0.0.1:43190 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:10:49.575058 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 29079
I20260807 04:10:49.583467 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:49.576062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.18.193.1:35139)
0807 04:10:49.576746 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:49.576753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:49.576778 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:10:49.576840 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:49.576846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:49.577052 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:49.577245 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:49.577686 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.578766 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.580075 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.580733 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.580875 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:49.582041 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:49.582050 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:49.582063 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:49.582613 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:49.582658 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:49.582683 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:49.582814 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:49.582975 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 04:10:49.583123 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":344,"threads_started":1}
I20260807 04:10:49.584996 29207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:49.670168 29212 heartbeater.cc:503] Master 127.18.193.62:38691 was elected leader, sending a full tablet report...
I20260807 04:10:49.671326 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 29079
W20260807 04:10:49.689814 29000 connection.cc:535] server connection from 127.18.193.1:46871 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 04:10:49.690205 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 29216
I20260807 04:10:49.698837 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:49.691229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.18.193.2:46795)
0807 04:10:49.691800 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:49.691807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:49.691835 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:10:49.691899 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:49.691905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:49.692139 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:49.692348 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:49.692834 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.693892 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.695190 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.695780 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.695973 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:49.697229 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:49.697238 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:49.697251 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:49.697876 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:49.697932 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:49.697953 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:49.698129 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:49.698274 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 04:10:49.698405 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":280,"threads_started":1}
I20260807 04:10:49.700294 29348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:49.786659 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 29216
I20260807 04:10:49.804836 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 29358
I20260807 04:10:49.813822 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:49.806032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.18.193.3:43471)
0807 04:10:49.806714 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:49.806721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:49.806752 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:10:49.806816 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:49.806824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:49.807045 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:49.807236 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:49.807639 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.808767 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.810045 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:49.810686 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:49.810787 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:49.812077 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:49.812086 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:49.812100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:49.812802 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:49.812855 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:49.812883 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:49.812984 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:49.813214 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 04:10:49.813407 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":374,"threads_started":1}
I20260807 04:10:49.815225 29485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:49.902281 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 29358
I20260807 04:10:49.920272 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 28982
I20260807 04:10:49.922143 29048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:50.029024 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 28982
2026-08-07T04:10:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2610 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T04:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:50Z Disabled control of system clock
I20260807 04:10:50.117707 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:36013
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:36013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 04:10:50.485630 29511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:50.485946 29511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:50.486099 29511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:50.493636 29511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:50.493763 29511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:50.493853 29511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:50.493960 29511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:50.494031 29511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:50.503629 29511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:36013
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:36013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:50.505117 29511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:50.506976 29511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:50.519506 29516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:50.519608 29519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:50.519641 29517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:50.520232 29511 server_base.cc:1034] running on GCE node
I20260807 04:10:50.521106 29511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:50.522607 29511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:50.523865 29511 hybrid_clock.cc:648] HybridClock initialized: now 1786075850523807 us; error 47 us; skew 500 ppm
I20260807 04:10:50.524279 29511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:50.526912 29511 webserver.cc:460] Webserver started at http://127.18.193.62:44739/ using document root <none> and password file <none>
I20260807 04:10:50.527663 29511 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:50.527786 29511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:50.528160 29511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:50.531297 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "c78a1d6011c642bd935114ba5ecd2cde"
format_stamp: "Formatted at 2026-08-07 04:10:50 on dist-test-slave-8n49"
server_key: "819f9789c94ec1aed6e74dbeba45b09a"
server_key_iv: "3ea7280ac34226c2163d0bc815b765a1"
server_key_version: "encryptionkey@0"
I20260807 04:10:50.532155 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "c78a1d6011c642bd935114ba5ecd2cde"
format_stamp: "Formatted at 2026-08-07 04:10:50 on dist-test-slave-8n49"
server_key: "819f9789c94ec1aed6e74dbeba45b09a"
server_key_iv: "3ea7280ac34226c2163d0bc815b765a1"
server_key_version: "encryptionkey@0"
I20260807 04:10:50.537814 29511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 04:10:50.541846 29525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:50.543394 29511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:10:50.543637 29511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "c78a1d6011c642bd935114ba5ecd2cde"
format_stamp: "Formatted at 2026-08-07 04:10:50 on dist-test-slave-8n49"
server_key: "819f9789c94ec1aed6e74dbeba45b09a"
server_key_iv: "3ea7280ac34226c2163d0bc815b765a1"
server_key_version: "encryptionkey@0"
I20260807 04:10:50.543900 29511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:50.550442 29511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:50.551312 29511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:50.551645 29511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:50.571739 29511 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:36013
I20260807 04:10:50.571777 29579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:36013 every 8 connection(s)
I20260807 04:10:50.573340 29511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:50.577328 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 29511
I20260807 04:10:50.577577 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:50.577651 29580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:50.578420 19204 external_mini_cluster.cc:1428] Setting key abb5bda3e364eb84fccd6794906f9ab0
I20260807 04:10:50.587929 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:50.580707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.18.193.62:36013)
0807 04:10:50.581848 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:50.581860 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:50.581903 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:10:50.584074 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:50.584097 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:50.584544 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:50.585039 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:50.586126 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:50.586132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:50.586145 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:50.586172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:50.586273 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:50.586302 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:50.586571 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:50.586707 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 04:10:50.586992 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":406,"threads_started":1}
I20260807 04:10:50.591744 29580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde: Bootstrap starting.
I20260807 04:10:50.594923 29580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:50.596298 29580 log.cc:824] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:50.599434 29580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde: No bootstrap required, opened a new log
I20260807 04:10:50.607380 29580 raft_consensus.cc:348] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c78a1d6011c642bd935114ba5ecd2cde" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 36013 } }
I20260807 04:10:50.607800 29580 raft_consensus.cc:374] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:50.607867 29580 raft_consensus.cc:729] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78a1d6011c642bd935114ba5ecd2cde, State: Initialized, Role: FOLLOWER
I20260807 04:10:50.608387 29580 consensus_queue.cc:260] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c78a1d6011c642bd935114ba5ecd2cde" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 36013 } }
I20260807 04:10:50.608666 29580 raft_consensus.cc:388] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:50.608783 29580 raft_consensus.cc:482] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:50.608952 29580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:50.611680 29580 raft_consensus.cc:504] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c78a1d6011c642bd935114ba5ecd2cde" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 36013 } }
I20260807 04:10:50.612157 29580 leader_election.cc:302] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c78a1d6011c642bd935114ba5ecd2cde; no voters:
I20260807 04:10:50.612900 29580 leader_election.cc:288] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:50.613041 29585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:50.613806 29585 raft_consensus.cc:686] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [term 1 LEADER]: Becoming Leader. State: Replica: c78a1d6011c642bd935114ba5ecd2cde, State: Running, Role: LEADER
I20260807 04:10:50.614423 29585 consensus_queue.cc:237] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c78a1d6011c642bd935114ba5ecd2cde" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 36013 } }
I20260807 04:10:50.615201 29580 sys_catalog.cc:564] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:50.617820 29586 sys_catalog.cc:455] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c78a1d6011c642bd935114ba5ecd2cde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c78a1d6011c642bd935114ba5ecd2cde" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 36013 } } }
I20260807 04:10:50.618180 29586 sys_catalog.cc:458] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:50.619338 29587 sys_catalog.cc:455] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [sys.catalog]: SysCatalogTable state changed. Reason: New leader c78a1d6011c642bd935114ba5ecd2cde. Latest consensus state: current_term: 1 leader_uuid: "c78a1d6011c642bd935114ba5ecd2cde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c78a1d6011c642bd935114ba5ecd2cde" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 36013 } } }
I20260807 04:10:50.619859 29587 sys_catalog.cc:458] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:50.621158 29594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:50.626775 29594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:50.635259 29594 catalog_manager.cc:1347] Generated new cluster ID: a97d4f1a7ab74c77a76937dd90ddf3be
I20260807 04:10:50.635394 29594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:50.661737 29594 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:50.662309 29594 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:50.676081 29594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c78a1d6011c642bd935114ba5ecd2cde: Generated new TSK 0
I20260807 04:10:50.677068 29594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:10:50.688333 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:36013
--builtin_ntp_servers=127.18.193.20:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 04:10:51.043439 29604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:51.043823 29604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:51.044004 29604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:51.051710 29604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:51.051888 29604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:51.052083 29604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:51.061720 29604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:51.063138 29604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:51.065002 29604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:51.079169 29609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:51.079119 29610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:51.079107 29612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:51.079912 29604 server_base.cc:1034] running on GCE node
I20260807 04:10:51.080655 29604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:51.081655 29604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:51.082901 29604 hybrid_clock.cc:648] HybridClock initialized: now 1786075851082839 us; error 75 us; skew 500 ppm
I20260807 04:10:51.083248 29604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:51.085778 29604 webserver.cc:460] Webserver started at http://127.18.193.1:38913/ using document root <none> and password file <none>
I20260807 04:10:51.086513 29604 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:51.086658 29604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:51.086992 29604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:51.090057 29604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "29b90a3c962745109d5906de905e82dd"
format_stamp: "Formatted at 2026-08-07 04:10:51 on dist-test-slave-8n49"
server_key: "a8f1bc902e45e7228cd072c5234994fc"
server_key_iv: "c2c43f37909d552d1bdd90d8438b6e66"
server_key_version: "encryptionkey@0"
I20260807 04:10:51.090915 29604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "29b90a3c962745109d5906de905e82dd"
format_stamp: "Formatted at 2026-08-07 04:10:51 on dist-test-slave-8n49"
server_key: "a8f1bc902e45e7228cd072c5234994fc"
server_key_iv: "c2c43f37909d552d1bdd90d8438b6e66"
server_key_version: "encryptionkey@0"
I20260807 04:10:51.096529 29604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 04:10:51.100199 29618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:51.101773 29604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:10:51.101974 29604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "29b90a3c962745109d5906de905e82dd"
format_stamp: "Formatted at 2026-08-07 04:10:51 on dist-test-slave-8n49"
server_key: "a8f1bc902e45e7228cd072c5234994fc"
server_key_iv: "c2c43f37909d552d1bdd90d8438b6e66"
server_key_version: "encryptionkey@0"
I20260807 04:10:51.102233 29604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:51.108214 29604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:51.109099 29604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:51.109385 29604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:51.110332 29604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:51.111831 29604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:51.111958 29604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:51.112106 29604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:51.112197 29604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:51.147795 29604 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:46867
I20260807 04:10:51.147833 29739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:46867 every 8 connection(s)
I20260807 04:10:51.149413 29604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:51.157446 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 29604
I20260807 04:10:51.157697 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:10:51.158351 19204 external_mini_cluster.cc:1428] Setting key 82db96ba046fcd08a6fa58ef0963bed6
I20260807 04:10:51.161384 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:36013
--builtin_ntp_servers=127.18.193.20:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 04:10:51.162457 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:51.152120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:38675 (local address 127.18.193.62:36013)
0807 04:10:51.152637 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:51.152644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:51.153855 (+ 1211us) server_negotiation.cc:407] Connection header received
0807 04:10:51.155579 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:51.155584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:51.155661 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:51.155765 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:51.161332 (+ 5567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:51.161341 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:51.161349 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:51.161375 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:51.161484 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:51.161510 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:51.161662 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:51.161952 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 04:10:51.162029 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":238,"threads_started":1}
I20260807 04:10:51.163319 29743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:51.152739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:36013 (local address 127.18.193.1:38675)
0807 04:10:51.153686 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:51.153727 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:51.155302 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:51.155977 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:51.155985 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:51.156078 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:51.160923 (+ 4845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:10:51.160926 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:51.160975 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:10:51.161119 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:51.161721 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:51.161732 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:51.161739 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:10:51.161991 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 04:10:51.162238 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":283,"threads_started":1}
I20260807 04:10:51.165038 29740 heartbeater.cc:347] Connected to a master server at 127.18.193.62:36013
I20260807 04:10:51.165393 29740 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:51.166163 29740 heartbeater.cc:511] Master 127.18.193.62:36013 requested a full tablet report, sending...
I20260807 04:10:51.168496 29543 ts_manager.cc:194] Registered new tserver with Master: 29b90a3c962745109d5906de905e82dd (127.18.193.1:46867)
W20260807 04:10:51.511710 29744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:51.512027 29744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:51.512214 29744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:51.520004 29744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:51.520115 29744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:51.520306 29744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:51.530311 29744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:51.531607 29744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:51.533468 29744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:51.546584 29750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:51.547021 29749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:51.547304 29752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:51.547745 29744 server_base.cc:1034] running on GCE node
I20260807 04:10:51.548482 29744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:51.549636 29744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:51.550876 29744 hybrid_clock.cc:648] HybridClock initialized: now 1786075851550799 us; error 60 us; skew 500 ppm
I20260807 04:10:51.551286 29744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:51.553960 29744 webserver.cc:460] Webserver started at http://127.18.193.2:33083/ using document root <none> and password file <none>
I20260807 04:10:51.554724 29744 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:51.554940 29744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:51.555328 29744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:51.558504 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "9e88647ea4a44ac09cfa2291b1fbc251"
format_stamp: "Formatted at 2026-08-07 04:10:51 on dist-test-slave-8n49"
server_key: "67bfabf8c82ab7faa9b7312f0061ce77"
server_key_iv: "b787e0ee994376d01d19f21d5246e63b"
server_key_version: "encryptionkey@0"
I20260807 04:10:51.559408 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "9e88647ea4a44ac09cfa2291b1fbc251"
format_stamp: "Formatted at 2026-08-07 04:10:51 on dist-test-slave-8n49"
server_key: "67bfabf8c82ab7faa9b7312f0061ce77"
server_key_iv: "b787e0ee994376d01d19f21d5246e63b"
server_key_version: "encryptionkey@0"
I20260807 04:10:51.565155 29744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:10:51.569123 29758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:51.570748 29744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:10:51.570981 29744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "9e88647ea4a44ac09cfa2291b1fbc251"
format_stamp: "Formatted at 2026-08-07 04:10:51 on dist-test-slave-8n49"
server_key: "67bfabf8c82ab7faa9b7312f0061ce77"
server_key_iv: "b787e0ee994376d01d19f21d5246e63b"
server_key_version: "encryptionkey@0"
I20260807 04:10:51.571250 29744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:51.577751 29744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:51.578658 29744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:51.578975 29744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:51.580001 29744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:51.581665 29744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:51.581799 29744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:51.581938 29744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:51.582032 29744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:51.616974 29744 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:42899
I20260807 04:10:51.617031 29877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:42899 every 8 connection(s)
I20260807 04:10:51.618366 29744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:51.621613 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 29744
I20260807 04:10:51.621845 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:10:51.622499 19204 external_mini_cluster.cc:1428] Setting key 4d9581d2e2009dd0839d1b052a4be45d
I20260807 04:10:51.625936 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:36013
--builtin_ntp_servers=127.18.193.20:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 04:10:51.627975 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:51.621137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:57329 (local address 127.18.193.62:36013)
0807 04:10:51.621379 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:51.621385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:51.623176 (+ 1791us) server_negotiation.cc:407] Connection header received
0807 04:10:51.625057 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:51.625061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:51.625122 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:51.625211 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:51.627091 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:51.627096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:51.627099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:51.627118 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:51.627155 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:51.627168 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:51.627293 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:51.627577 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 04:10:51.627680 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 04:10:51.629328 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:51.621754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:36013 (local address 127.18.193.2:57329)
0807 04:10:51.622967 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:51.623018 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:51.624712 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:51.625525 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:51.625540 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:51.625676 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:51.626669 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:10:51.626674 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:51.626730 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:10:51.626858 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:51.627300 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:51.627313 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:51.627325 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 04:10:51.627660 (+ 335us) client_negotiation.cc:240] Negotiation successful
0807 04:10:51.627974 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":340,"threads_started":1}
I20260807 04:10:51.631407 29878 heartbeater.cc:347] Connected to a master server at 127.18.193.62:36013
I20260807 04:10:51.631909 29878 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:51.632936 29878 heartbeater.cc:511] Master 127.18.193.62:36013 requested a full tablet report, sending...
I20260807 04:10:51.635095 29543 ts_manager.cc:194] Registered new tserver with Master: 9e88647ea4a44ac09cfa2291b1fbc251 (127.18.193.2:42899)
W20260807 04:10:51.978084 29881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:51.978377 29881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:51.978535 29881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:51.986042 29881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:51.986161 29881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:51.986338 29881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:51.995735 29881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:51.997130 29881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:51.998894 29881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:52.013747 29888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:52.013824 29890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:52.013782 29887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:52.014971 29881 server_base.cc:1034] running on GCE node
I20260807 04:10:52.015699 29881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:52.016996 29881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:52.018239 29881 hybrid_clock.cc:648] HybridClock initialized: now 1786075852018187 us; error 52 us; skew 500 ppm
I20260807 04:10:52.018644 29881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:10:52.021631 29881 webserver.cc:460] Webserver started at http://127.18.193.3:34517/ using document root <none> and password file <none>
I20260807 04:10:52.022511 29881 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:52.022612 29881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:52.022967 29881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:52.026477 29881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "1f04c988824c4ad3adbb77967a6f5f93"
format_stamp: "Formatted at 2026-08-07 04:10:52 on dist-test-slave-8n49"
server_key: "979ea5c5057a32a59b8dbae10496938d"
server_key_iv: "2068bd213e0b03c7bead64ea0b0baa75"
server_key_version: "encryptionkey@0"
I20260807 04:10:52.027359 29881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "1f04c988824c4ad3adbb77967a6f5f93"
format_stamp: "Formatted at 2026-08-07 04:10:52 on dist-test-slave-8n49"
server_key: "979ea5c5057a32a59b8dbae10496938d"
server_key_iv: "2068bd213e0b03c7bead64ea0b0baa75"
server_key_version: "encryptionkey@0"
I20260807 04:10:52.033550 29881 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 04:10:52.037492 29896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:52.039309 29881 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:10:52.039525 29881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "1f04c988824c4ad3adbb77967a6f5f93"
format_stamp: "Formatted at 2026-08-07 04:10:52 on dist-test-slave-8n49"
server_key: "979ea5c5057a32a59b8dbae10496938d"
server_key_iv: "2068bd213e0b03c7bead64ea0b0baa75"
server_key_version: "encryptionkey@0"
I20260807 04:10:52.039779 29881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:52.046101 29881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:52.046962 29881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:52.047253 29881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:52.048456 29881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:52.050084 29881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:52.050208 29881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:52.050352 29881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:52.050446 29881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:52.086309 29881 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33337
I20260807 04:10:52.086407 30014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33337 every 8 connection(s)
I20260807 04:10:52.087718 29881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:52.096194 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 29881
I20260807 04:10:52.096344 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:52.090720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:38965 (local address 127.18.193.62:36013)
0807 04:10:52.090914 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:52.090918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:52.092304 (+ 1386us) server_negotiation.cc:407] Connection header received
0807 04:10:52.094083 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:52.094087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:52.094143 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:52.094238 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:52.095556 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:52.095560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:52.095563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:52.095599 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:52.095630 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:52.095643 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:52.095770 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:52.095953 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 04:10:52.096026 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:10:52.096402 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:10:52.096918 19204 external_mini_cluster.cc:1428] Setting key bdb48fef2f50188fb1a790cb2ebcb9a7
I20260807 04:10:52.097256 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:52.091204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:36013 (local address 127.18.193.3:38965)
0807 04:10:52.092140 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:52.092175 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:52.093802 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:52.094442 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:52.094450 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:52.094538 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:52.095264 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:10:52.095267 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:52.095305 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:10:52.095389 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:52.095736 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:52.095744 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:52.095749 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:10:52.095972 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 04:10:52.096158 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":307,"threads_started":1}
I20260807 04:10:52.098898 30015 heartbeater.cc:347] Connected to a master server at 127.18.193.62:36013
I20260807 04:10:52.099223 30015 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:52.099877 30015 heartbeater.cc:511] Master 127.18.193.62:36013 requested a full tablet report, sending...
I20260807 04:10:52.101572 29542 ts_manager.cc:194] Registered new tserver with Master: 1f04c988824c4ad3adbb77967a6f5f93 (127.18.193.3:33337)
I20260807 04:10:52.113478 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 04:10:52.120043 29742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:10:52.118403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40492 (local address 127.18.193.62:36013)
0807 04:10:52.118614 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:52.118619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:52.119165 (+ 546us) server_negotiation.cc:407] Connection header received
0807 04:10:52.119342 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:52.119347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:52.119417 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:52.119514 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:52.119840 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40492: BlockingRecv error: recv got EOF from 127.0.0.1:40492 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:10:52.120692 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 29604
I20260807 04:10:52.124975 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:52.122061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.18.193.1:46867)
0807 04:10:52.122668 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:52.122675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:52.122701 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:10:52.122761 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:52.122768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:52.122987 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:52.123118 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:52.124146 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:52.124153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:52.124165 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:52.124186 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:52.124229 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:52.124243 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:52.124353 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:52.124629 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 04:10:52.124718 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":291,"threads_started":1}
I20260807 04:10:52.126292 29734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:52.209569 29740 heartbeater.cc:503] Master 127.18.193.62:36013 was elected leader, sending a full tablet report...
I20260807 04:10:52.211009 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 29604
W20260807 04:10:52.230098 29529 connection.cc:535] server connection from 127.18.193.1:38675 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 04:10:52.230394 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 29744
I20260807 04:10:52.234459 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:52.231351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.18.193.2:42899)
0807 04:10:52.231964 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:52.231971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:52.232000 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:10:52.232076 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:52.232084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:52.232307 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:52.232456 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:52.233693 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:52.233700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:52.233712 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:52.233738 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:52.233790 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:52.233806 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:52.233927 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:52.234126 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 04:10:52.234215 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":331,"threads_started":1}
I20260807 04:10:52.236057 29872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:52.321028 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 29744
I20260807 04:10:52.341249 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 29881
I20260807 04:10:52.345121 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:52.342196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.18.193.3:33337)
0807 04:10:52.342833 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:52.342840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:52.342871 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:10:52.342934 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:52.342940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:52.343163 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:52.343289 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:52.344309 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:52.344315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:52.344326 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:10:52.344337 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:52.344379 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:10:52.344396 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:52.344523 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:52.344664 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 04:10:52.344781 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":273,"threads_started":1}
I20260807 04:10:52.346483 30010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:52.430941 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 29881
I20260807 04:10:52.452800 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 29511
I20260807 04:10:52.454351 29573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:52.562979 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 29511
2026-08-07T04:10:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2533 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:10:52 dist-test-slave-8n49 krb5kdc[30034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:10:52 dist-test-slave-8n49 krb5kdc[30034](info): set up 2 sockets
Aug 07 04:10:52 dist-test-slave-8n49 krb5kdc[30034](info): commencing operation
krb5kdc: starting...
W20260807 04:10:53.754842 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:10:53 dist-test-slave-8n49 krb5kdc[30034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:10:54.062764 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:45701
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:45701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:54.416079 30047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:54.416452 30047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:54.416648 30047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:54.424172 30047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:10:54.424335 30047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:54.424453 30047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:54.424590 30047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:10:54.424671 30047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:10:54.435070 30047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:45701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:45701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:54.436524 30047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:54.437928 30047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:54.451663 30054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:54.452992 30053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:54.453446 30056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:54.455207 30047 server_base.cc:1034] running on GCE node
I20260807 04:10:54.456051 30047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:54.457648 30047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:54.458880 30047 hybrid_clock.cc:648] HybridClock initialized: now 1786075854458827 us; error 33 us; skew 500 ppm
I20260807 04:10:54.459281 30047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:54 dist-test-slave-8n49 krb5kdc[30034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:54.462782 30047 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:10:54.464515 30047 webserver.cc:460] Webserver started at http://127.18.193.62:46601/ using document root <none> and password file <none>
I20260807 04:10:54.465364 30047 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:54.465504 30047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:54.465876 30047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:54.469118 30047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "516ec7aa20dc4b2d82c7a90923aefe0e"
format_stamp: "Formatted at 2026-08-07 04:10:54 on dist-test-slave-8n49"
server_key: "3d0ee0452e911b2bfddbfbe990f8bfb7"
server_key_iv: "32a43070d2b2d5d69022f0dc56e922c5"
server_key_version: "encryptionkey@0"
I20260807 04:10:54.469942 30047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "516ec7aa20dc4b2d82c7a90923aefe0e"
format_stamp: "Formatted at 2026-08-07 04:10:54 on dist-test-slave-8n49"
server_key: "3d0ee0452e911b2bfddbfbe990f8bfb7"
server_key_iv: "32a43070d2b2d5d69022f0dc56e922c5"
server_key_version: "encryptionkey@0"
I20260807 04:10:54.475572 30047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:10:54.479137 30063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:54.480618 30047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:54.480815 30047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "516ec7aa20dc4b2d82c7a90923aefe0e"
format_stamp: "Formatted at 2026-08-07 04:10:54 on dist-test-slave-8n49"
server_key: "3d0ee0452e911b2bfddbfbe990f8bfb7"
server_key_iv: "32a43070d2b2d5d69022f0dc56e922c5"
server_key_version: "encryptionkey@0"
I20260807 04:10:54.481002 30047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:54.506907 30047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:54.511734 30047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:54.512079 30047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:54.533821 30047 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:45701
I20260807 04:10:54.533893 30118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:45701 every 8 connection(s)
I20260807 04:10:54.535264 30047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:10:54.539182 30119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:54.542935 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 30047
I20260807 04:10:54.543210 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:10:54.543908 19204 external_mini_cluster.cc:1428] Setting key 1724ca6f04bb3101d7f1d1c3bad2959d
I20260807 04:10:54.551994 30119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e: Bootstrap starting.
I20260807 04:10:54.556059 30119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:54.557458 30119 log.cc:824] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:54.561357 30119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e: No bootstrap required, opened a new log
Aug 07 04:10:54 dist-test-slave-8n49 krb5kdc[30034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:54.572928 30122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:54.546397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.18.193.62:45701)
0807 04:10:54.547445 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:54.547462 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:54.547953 (+ 491us) server_negotiation.cc:407] Connection header received
0807 04:10:54.550542 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:54.550572 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:54.551131 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:54.551610 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:54.553215 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:54.555323 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:54.556742 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:54.557830 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:54.558060 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:54.563214 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:54.563226 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:54.563247 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:54.563309 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:54.567740 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:54.568598 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:54.568613 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:54.568624 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:54.568853 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:54.569305 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:54.569309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:54.569313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:54.570129 (+ 816us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:54.570483 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:54.570886 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 04:10:54.571399 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":505,"threads_started":1}
I20260807 04:10:54.572641 30119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45701 } }
I20260807 04:10:54.573146 30119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:54.573249 30119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 516ec7aa20dc4b2d82c7a90923aefe0e, State: Initialized, Role: FOLLOWER
I20260807 04:10:54.573865 30119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45701 } }
I20260807 04:10:54.574157 30119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:10:54.574311 30119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:10:54.574501 30119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:54.577623 30119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45701 } }
I20260807 04:10:54.578178 30119 leader_election.cc:302] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 516ec7aa20dc4b2d82c7a90923aefe0e; no voters:
I20260807 04:10:54.578989 30119 leader_election.cc:288] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:10:54.579144 30124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:54.579903 30124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [term 1 LEADER]: Becoming Leader. State: Replica: 516ec7aa20dc4b2d82c7a90923aefe0e, State: Running, Role: LEADER
I20260807 04:10:54.580595 30124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45701 } }
I20260807 04:10:54.581243 30119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:10:54.583340 30130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 516ec7aa20dc4b2d82c7a90923aefe0e. Latest consensus state: current_term: 1 leader_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45701 } } }
I20260807 04:10:54.583827 30130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:54.584122 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "516ec7aa20dc4b2d82c7a90923aefe0e" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45701 } } }
I20260807 04:10:54.584667 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e [sys.catalog]: This master's current role is: LEADER
I20260807 04:10:54.591532 30133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:10:54.597574 30133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:10:54.606984 30133 catalog_manager.cc:1347] Generated new cluster ID: c1bbaffbed3c41a582d076619a52b473
I20260807 04:10:54.607141 30133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:10:54.627465 30133 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:10:54.629045 30133 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:10:54.636679 30133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e: Generated new TSK 0
I20260807 04:10:54.637620 30133 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:10:54.712467 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:45701
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:55.068317 30149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:55.068730 30149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:55.068924 30149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:55.076653 30149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:55.076773 30149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:55.077014 30149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:10:55.087604 30149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:45701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:55.089078 30149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:55.090415 30149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:55.103909 30155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:55.103900 30156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:55.103967 30158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:55.104900 30149 server_base.cc:1034] running on GCE node
I20260807 04:10:55.105562 30149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:55.106537 30149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:55.107793 30149 hybrid_clock.cc:648] HybridClock initialized: now 1786075855107748 us; error 59 us; skew 500 ppm
I20260807 04:10:55.108155 30149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:55 dist-test-slave-8n49 krb5kdc[30034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:55.111634 30149 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:10:55.113329 30149 webserver.cc:460] Webserver started at http://127.18.193.1:46559/ using document root <none> and password file <none>
I20260807 04:10:55.114037 30149 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:55.114143 30149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:55.114418 30149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:55.117558 30149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "80c4766d875b4a26b6ad2e9b3fae95e5"
format_stamp: "Formatted at 2026-08-07 04:10:55 on dist-test-slave-8n49"
server_key: "683b57d8f2647af596e5eaaf117687f9"
server_key_iv: "71e433b5fdb4d0adff8cd1d15015365c"
server_key_version: "encryptionkey@0"
I20260807 04:10:55.118316 30149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "80c4766d875b4a26b6ad2e9b3fae95e5"
format_stamp: "Formatted at 2026-08-07 04:10:55 on dist-test-slave-8n49"
server_key: "683b57d8f2647af596e5eaaf117687f9"
server_key_iv: "71e433b5fdb4d0adff8cd1d15015365c"
server_key_version: "encryptionkey@0"
I20260807 04:10:55.123840 30149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:10:55.127869 30165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:55.129438 30149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:10:55.129649 30149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "80c4766d875b4a26b6ad2e9b3fae95e5"
format_stamp: "Formatted at 2026-08-07 04:10:55 on dist-test-slave-8n49"
server_key: "683b57d8f2647af596e5eaaf117687f9"
server_key_iv: "71e433b5fdb4d0adff8cd1d15015365c"
server_key_version: "encryptionkey@0"
I20260807 04:10:55.129861 30149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:55.152020 30149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:55.156963 30149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:55.157297 30149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:55.158373 30149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:55.159785 30149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:55.159863 30149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:55.160043 30149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:55.160107 30149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:55.194859 30149 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:46785
I20260807 04:10:55.194957 30287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:46785 every 8 connection(s)
I20260807 04:10:55.196219 30149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:10:55.196691 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 30149
I20260807 04:10:55.196935 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:10:55.197790 19204 external_mini_cluster.cc:1428] Setting key 42117df2d84e50dfbccfc0853b5cadd3
Aug 07 04:10:55 dist-test-slave-8n49 krb5kdc[30034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:55.218806 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:55.198991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:57667 (local address 127.18.193.62:45701)
0807 04:10:55.199445 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:55.199450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:55.200734 (+ 1284us) server_negotiation.cc:407] Connection header received
0807 04:10:55.202595 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:55.202603 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:55.202696 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:55.202819 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:55.204317 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.205477 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.207118 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.207532 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.207672 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:55.211463 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:55.211473 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:55.211481 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:55.211529 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:55.215307 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:55.216079 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:55.216087 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:55.216093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:55.216246 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:55.216718 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:55.216726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:55.216731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:55.217133 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:55.217313 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:55.218053 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 04:10:55.218317 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":244,"threads_started":1}
I20260807 04:10:55.219442 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:55.199688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:45701 (local address 127.18.193.1:57667)
0807 04:10:55.200572 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:55.200622 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:55.202317 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:55.203018 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:55.203027 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:55.203579 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:55.204145 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:55.204162 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.205653 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.205657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:55.206946 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:55.206959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.207784 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.207792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:55.208030 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:55.208786 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:55.208815 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:55.211221 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:55.215468 (+ 4247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:55.215475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:55.215490 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:55.215939 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:55.216359 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:55.216363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:55.216365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:55.216571 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:55.217279 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:55.217285 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:55.217638 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 04:10:55.217868 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 04:10:55.218279 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":362,"threads_started":1}
I20260807 04:10:55.221081 30288 heartbeater.cc:347] Connected to a master server at 127.18.193.62:45701
I20260807 04:10:55.221427 30288 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:55.222398 30288 heartbeater.cc:511] Master 127.18.193.62:45701 requested a full tablet report, sending...
I20260807 04:10:55.224895 30083 ts_manager.cc:194] Registered new tserver with Master: 80c4766d875b4a26b6ad2e9b3fae95e5 (127.18.193.1:46785)
I20260807 04:10:55.226372 30083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:57667
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:10:55.265417 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:45701
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:55.620030 30295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:55.620332 30295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:55.620517 30295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:55.628057 30295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:55.628170 30295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:55.628283 30295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:10:55.638657 30295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:45701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:55.639938 30295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:55.641309 30295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:55.655229 30300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:55.655200 30303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:55.655200 30301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:55.656801 30295 server_base.cc:1034] running on GCE node
I20260807 04:10:55.657472 30295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:55.658452 30295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:55.659670 30295 hybrid_clock.cc:648] HybridClock initialized: now 1786075855659627 us; error 35 us; skew 500 ppm
I20260807 04:10:55.659971 30295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:55 dist-test-slave-8n49 krb5kdc[30034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:55.663259 30295 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:10:55.664745 30295 webserver.cc:460] Webserver started at http://127.18.193.2:37273/ using document root <none> and password file <none>
I20260807 04:10:55.665441 30295 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:55.665542 30295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:55.665798 30295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:55.668908 30295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "110f917e652f4eb8a823716fc2344c02"
format_stamp: "Formatted at 2026-08-07 04:10:55 on dist-test-slave-8n49"
server_key: "2170c4b198b07a82a86f0be7d4ba8198"
server_key_iv: "8f2d75b88d7eaa0f30799aea21e1882c"
server_key_version: "encryptionkey@0"
I20260807 04:10:55.669641 30295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "110f917e652f4eb8a823716fc2344c02"
format_stamp: "Formatted at 2026-08-07 04:10:55 on dist-test-slave-8n49"
server_key: "2170c4b198b07a82a86f0be7d4ba8198"
server_key_iv: "8f2d75b88d7eaa0f30799aea21e1882c"
server_key_version: "encryptionkey@0"
I20260807 04:10:55.674983 30295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:10:55.678550 30310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:55.679946 30295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:10:55.680094 30295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "110f917e652f4eb8a823716fc2344c02"
format_stamp: "Formatted at 2026-08-07 04:10:55 on dist-test-slave-8n49"
server_key: "2170c4b198b07a82a86f0be7d4ba8198"
server_key_iv: "8f2d75b88d7eaa0f30799aea21e1882c"
server_key_version: "encryptionkey@0"
I20260807 04:10:55.680290 30295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:55.710501 30295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:55.715184 30295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:55.715461 30295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:55.716429 30295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:55.717918 30295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:55.717994 30295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:55.718091 30295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:55.718139 30295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:55.753973 30295 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:38767
I20260807 04:10:55.754070 30429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:38767 every 8 connection(s)
I20260807 04:10:55.755333 30295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:10:55.763932 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 30295
I20260807 04:10:55.764170 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:10:55.764684 19204 external_mini_cluster.cc:1428] Setting key 0b5aee9bb29a50a8824521cdfe90abb2
Aug 07 04:10:55 dist-test-slave-8n49 krb5kdc[30034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:55.777680 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:55.758209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:55135 (local address 127.18.193.62:45701)
0807 04:10:55.758776 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:55.758781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:55.759621 (+ 840us) server_negotiation.cc:407] Connection header received
0807 04:10:55.761600 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:55.761605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:55.761706 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:55.761830 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:55.763970 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.764973 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.767123 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.767592 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.767739 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:55.771656 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:55.771662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:55.771665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:55.771701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:55.774360 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:55.775147 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:55.775153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:55.775156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:55.775296 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:55.775766 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:55.775774 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:55.775777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:55.776060 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:55.776216 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:55.777010 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 04:10:55.777251 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":347,"threads_started":1}
I20260807 04:10:55.778741 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:55.758666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:45701 (local address 127.18.193.2:55135)
0807 04:10:55.759432 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:55.759473 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:55.761303 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:55.762091 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:55.762099 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:55.762759 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:55.763758 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:55.763789 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.765152 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.765158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:55.766930 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:55.766954 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:55.767856 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:55.767864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:55.768175 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:55.768966 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:55.768989 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:55.771378 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:55.774520 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:55.774530 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:55.774549 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:55.775016 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:55.775412 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:55.775417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:55.775421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:55.775646 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:55.776190 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:55.776198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:55.776629 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 04:10:55.776927 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 04:10:55.777513 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":321,"threads_started":1}
I20260807 04:10:55.780536 30430 heartbeater.cc:347] Connected to a master server at 127.18.193.62:45701
I20260807 04:10:55.780973 30430 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:55.781929 30430 heartbeater.cc:511] Master 127.18.193.62:45701 requested a full tablet report, sending...
I20260807 04:10:55.783829 30083 ts_manager.cc:194] Registered new tserver with Master: 110f917e652f4eb8a823716fc2344c02 (127.18.193.2:38767)
I20260807 04:10:55.785296 30083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:55135
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:10:55.832347 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:45701
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:10:56.175036 30436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:10:56.175319 30436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:10:56.175415 30436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:10:56.183039 30436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:10:56.183153 30436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:10:56.183265 30436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:10:56.193539 30436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:45701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:10:56.195060 30436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:10:56.196769 30436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:10:56.210788 30441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:56.210763 30442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:10:56.210726 30444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:10:56.211565 30436 server_base.cc:1034] running on GCE node
I20260807 04:10:56.212227 30436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:10:56.213245 30436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:10:56.214504 30436 hybrid_clock.cc:648] HybridClock initialized: now 1786075856214433 us; error 51 us; skew 500 ppm
I20260807 04:10:56.214833 30436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:10:56 dist-test-slave-8n49 krb5kdc[30034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:10:56.222039 30436 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:10:56.223590 30436 webserver.cc:460] Webserver started at http://127.18.193.3:34903/ using document root <none> and password file <none>
I20260807 04:10:56.224301 30436 fs_manager.cc:359] Metadata directory not provided
I20260807 04:10:56.224464 30436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:10:56.224808 30436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:10:56.228092 30436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "7e7809f7496445e1a9792823a6216bdf"
format_stamp: "Formatted at 2026-08-07 04:10:56 on dist-test-slave-8n49"
server_key: "de5455e064475bbcb90bd60b05b6a1f5"
server_key_iv: "3b3c55650fd6e0994d3cb36ee41c0fbc"
server_key_version: "encryptionkey@0"
I20260807 04:10:56.228941 30436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "7e7809f7496445e1a9792823a6216bdf"
format_stamp: "Formatted at 2026-08-07 04:10:56 on dist-test-slave-8n49"
server_key: "de5455e064475bbcb90bd60b05b6a1f5"
server_key_iv: "3b3c55650fd6e0994d3cb36ee41c0fbc"
server_key_version: "encryptionkey@0"
I20260807 04:10:56.229817 30288 heartbeater.cc:503] Master 127.18.193.62:45701 was elected leader, sending a full tablet report...
I20260807 04:10:56.234495 30436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:10:56.237989 30451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:56.239310 30436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:10:56.239495 30436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "7e7809f7496445e1a9792823a6216bdf"
format_stamp: "Formatted at 2026-08-07 04:10:56 on dist-test-slave-8n49"
server_key: "de5455e064475bbcb90bd60b05b6a1f5"
server_key_iv: "3b3c55650fd6e0994d3cb36ee41c0fbc"
server_key_version: "encryptionkey@0"
I20260807 04:10:56.239706 30436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:10:56.272320 30436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:10:56.276386 30436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:10:56.276767 30436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:10:56.277769 30436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:10:56.279103 30436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:10:56.279208 30436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:56.279371 30436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:10:56.279443 30436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:10:56.315191 30436 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33187
I20260807 04:10:56.315222 30570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33187 every 8 connection(s)
I20260807 04:10:56.316524 30436 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:10:56.321113 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 30436
I20260807 04:10:56.321333 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:10:56.321887 19204 external_mini_cluster.cc:1428] Setting key f47e7fca4e6d71969321fc212f9c8bdf
Aug 07 04:10:56 dist-test-slave-8n49 krb5kdc[30034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:10:56.338737 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.319250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:53837 (local address 127.18.193.62:45701)
0807 04:10:56.319741 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.319746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.320957 (+ 1211us) server_negotiation.cc:407] Connection header received
0807 04:10:56.323384 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.323390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.323497 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.323618 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:56.325479 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.326257 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.327831 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.328242 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.328376 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.332224 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:56.332230 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:56.332233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:56.332272 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:56.335378 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:56.336151 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:56.336158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:56.336167 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:56.336285 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:56.336820 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:56.336827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:56.336835 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:56.337127 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:56.337291 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.338074 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.338310 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":228,"threads_started":1}
I20260807 04:10:56.339798 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.319888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:45701 (local address 127.18.193.3:53837)
0807 04:10:56.320742 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:56.320804 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:56.323061 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:56.323867 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:56.323875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:56.324635 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:10:56.325303 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.325322 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.326432 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.326435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.327673 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.327686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.328498 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.328506 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.328775 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.329511 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:10:56.329554 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:10:56.331948 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:10:56.335535 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:56.335545 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:56.335565 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:56.336009 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:56.336401 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:10:56.336406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:10:56.336410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:10:56.336697 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:10:56.337265 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:10:56.337273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:10:56.337677 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 04:10:56.337960 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 04:10:56.338549 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":299,"threads_started":1}
I20260807 04:10:56.341370 30571 heartbeater.cc:347] Connected to a master server at 127.18.193.62:45701
I20260807 04:10:56.341756 30571 heartbeater.cc:464] Registering TS with master...
I20260807 04:10:56.342535 30571 heartbeater.cc:511] Master 127.18.193.62:45701 requested a full tablet report, sending...
I20260807 04:10:56.344488 30083 ts_manager.cc:194] Registered new tserver with Master: 7e7809f7496445e1a9792823a6216bdf (127.18.193.3:33187)
I20260807 04:10:56.345461 30083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:53837
I20260807 04:10:56.350323 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:10:56.369998 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.354926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.18.193.62:45701)
0807 04:10:56.355112 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.355116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.355651 (+ 535us) server_negotiation.cc:407] Connection header received
0807 04:10:56.355798 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.355801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.355868 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.355966 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:56.357611 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.358373 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.359661 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.360059 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.360168 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.362960 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:56.362968 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:56.362972 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:56.363035 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:56.367055 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:56.367865 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:56.367870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:56.367872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:56.368029 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:56.368427 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:56.368431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:56.368433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:56.369045 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:56.369268 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.369570 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.369766 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:10:56.374512 30083 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:10:56.381769 30083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:10:56.405627 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.395929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:46785 (local address 127.0.0.1:49242)
0807 04:10:56.396751 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:56.396796 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:56.396965 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:56.398290 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:56.398296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:56.398321 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:56.398622 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.398630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.400010 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.400019 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.402056 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.402069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.404278 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.404284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.404604 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.404678 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 04:10:56.404795 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 04:10:56.405087 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":18,"thread_start_us":6433,"threads_started":1}
I20260807 04:10:56.406287 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.396699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.18.193.1:46785)
0807 04:10:56.397476 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.397483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.397508 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:10:56.397583 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.397588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.397765 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.397976 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:56.398814 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.399863 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.402221 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.403817 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.405441 (+ 1624us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.405702 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.405780 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.405962 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":476,"threads_started":1}
I20260807 04:10:56.410552 30217 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7c98abb19048ea95bc30b68c7ffe06 (DEFAULT_TABLE table=test-table [id=3db6fcbdbf3941839edaec6186ea9121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:56.412108 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.395973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:38767 (local address 127.0.0.1:39422)
0807 04:10:56.402648 (+ 6675us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:56.402669 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:56.402834 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:56.404031 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:56.404035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:56.404063 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:56.406079 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.406095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.407933 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.407939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.409782 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.409795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.411238 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.411244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.411425 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.411459 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 04:10:56.411555 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 04:10:56.411714 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6543,"mutex_wait_us":5,"thread_start_us":416,"threads_started":1}
I20260807 04:10:56.412125 30217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7c98abb19048ea95bc30b68c7ffe06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:56.412184 30586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.396162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.18.193.2:38767)
0807 04:10:56.403368 (+ 7206us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.403377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.403407 (+ 30us) server_negotiation.cc:407] Connection header received
0807 04:10:56.403503 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.403511 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.403802 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.406145 (+ 2343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:56.406458 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.407789 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.409941 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.411017 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.411161 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.411354 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.411637 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.411817 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7034,"thread_start_us":6888,"threads_started":1}
I20260807 04:10:56.413156 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.404056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33187 (local address 127.0.0.1:37100)
0807 04:10:56.405882 (+ 1826us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:56.405899 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:56.405996 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:56.406681 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:56.406685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:56.406716 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:56.406882 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.406893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.408739 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.408746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.410627 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.410642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.412417 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.412423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.412625 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.412661 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:10:56.412754 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:10:56.412892 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1718,"thread_start_us":1560,"threads_started":1}
I20260807 04:10:56.413914 30588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.403327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.18.193.3:33187)
0807 04:10:56.404054 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.404062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.406137 (+ 2075us) server_negotiation.cc:407] Connection header received
0807 04:10:56.406246 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.406255 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.406495 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.406702 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:56.407023 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.408500 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.410804 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.412212 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.412347 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.412612 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.413190 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.413528 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":424,"threads_started":1}
I20260807 04:10:56.416011 30360 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7c98abb19048ea95bc30b68c7ffe06 (DEFAULT_TABLE table=test-table [id=3db6fcbdbf3941839edaec6186ea9121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:56.417223 30360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7c98abb19048ea95bc30b68c7ffe06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:56.417757 30502 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7c98abb19048ea95bc30b68c7ffe06 (DEFAULT_TABLE table=test-table [id=3db6fcbdbf3941839edaec6186ea9121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:10:56.419245 30502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7c98abb19048ea95bc30b68c7ffe06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:10:56.429417 30590 tablet_bootstrap.cc:492] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Bootstrap starting.
I20260807 04:10:56.430284 30591 tablet_bootstrap.cc:492] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Bootstrap starting.
I20260807 04:10:56.433859 30590 tablet_bootstrap.cc:654] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:56.434269 30592 tablet_bootstrap.cc:492] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Bootstrap starting.
I20260807 04:10:56.435527 30590 log.cc:824] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:56.436299 30591 tablet_bootstrap.cc:654] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:56.437641 30592 tablet_bootstrap.cc:654] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Neither blocks nor log segments found. Creating new log.
I20260807 04:10:56.437870 30591 log.cc:824] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:56.438831 30592 log.cc:824] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Log is configured to *not* fsync() on all Append() calls
I20260807 04:10:56.440671 30590 tablet_bootstrap.cc:492] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: No bootstrap required, opened a new log
I20260807 04:10:56.440984 30590 ts_tablet_manager.cc:1397] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260807 04:10:56.441708 30591 tablet_bootstrap.cc:492] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: No bootstrap required, opened a new log
I20260807 04:10:56.441947 30592 tablet_bootstrap.cc:492] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: No bootstrap required, opened a new log
I20260807 04:10:56.441974 30591 ts_tablet_manager.cc:1397] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 04:10:56.442205 30592 ts_tablet_manager.cc:1397] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:10:56.450331 30591 raft_consensus.cc:348] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.450766 30591 raft_consensus.cc:374] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:56.450870 30591 raft_consensus.cc:729] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110f917e652f4eb8a823716fc2344c02, State: Initialized, Role: FOLLOWER
I20260807 04:10:56.451620 30591 consensus_queue.cc:260] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.453135 30430 heartbeater.cc:503] Master 127.18.193.62:45701 was elected leader, sending a full tablet report...
I20260807 04:10:56.453686 30591 ts_tablet_manager.cc:1428] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260807 04:10:56.452620 30590 raft_consensus.cc:348] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.454186 30590 raft_consensus.cc:374] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:56.453866 30592 raft_consensus.cc:348] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.454320 30590 raft_consensus.cc:729] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c4766d875b4a26b6ad2e9b3fae95e5, State: Initialized, Role: FOLLOWER
I20260807 04:10:56.454362 30592 raft_consensus.cc:374] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:10:56.454475 30592 raft_consensus.cc:729] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7809f7496445e1a9792823a6216bdf, State: Initialized, Role: FOLLOWER
I20260807 04:10:56.455096 30590 consensus_queue.cc:260] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.455142 30592 consensus_queue.cc:260] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.456895 30571 heartbeater.cc:503] Master 127.18.193.62:45701 was elected leader, sending a full tablet report...
I20260807 04:10:56.457298 30592 ts_tablet_manager.cc:1428] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 04:10:56.457553 30590 ts_tablet_manager.cc:1428] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
W20260807 04:10:56.507558 30431 tablet.cc:2369] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:10:56.568325 30572 tablet.cc:2369] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:10:56.699318 30290 tablet.cc:2369] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:10:56.795138 30598 raft_consensus.cc:482] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:10:56.795444 30598 raft_consensus.cc:504] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.797286 30598 leader_election.cc:288] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 110f917e652f4eb8a823716fc2344c02 (127.18.193.2:38767), 7e7809f7496445e1a9792823a6216bdf (127.18.193.3:33187)
I20260807 04:10:56.804821 30588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.797856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:56019 (local address 127.18.193.3:33187)
0807 04:10:56.798071 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.798077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.798919 (+ 842us) server_negotiation.cc:407] Connection header received
0807 04:10:56.799117 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.799121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.799181 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.799291 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:56.800049 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.800870 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.802978 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.803783 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.803984 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.804056 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.804391 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.804522 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:10:56.804828 30586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.797781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:46411 (local address 127.18.193.2:38767)
0807 04:10:56.798014 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.798022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.798828 (+ 806us) server_negotiation.cc:407] Connection header received
0807 04:10:56.799060 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.799068 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.799151 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.799221 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:10:56.799868 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.800711 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.802865 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.803765 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.803942 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.804035 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.804382 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.804527 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:10:56.804979 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.797729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:38767 (local address 127.18.193.1:46411)
0807 04:10:56.798702 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:56.798718 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:56.798886 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:56.799279 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:56.799284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:56.799308 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:56.799715 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.799727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.800855 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.800860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.802683 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.802696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.804010 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.804017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.804193 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.804243 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 04:10:56.804354 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 04:10:56.804513 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":34,"thread_start_us":586,"threads_started":1}
I20260807 04:10:56.805438 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.797850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33187 (local address 127.18.193.1:56019)
0807 04:10:56.798757 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0807 04:10:56.798798 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:10:56.798940 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:10:56.799335 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:10:56.799340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:10:56.799370 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:10:56.799836 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.799850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.801018 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.801022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.802751 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:10:56.802799 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.804059 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.804067 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:10:56.804206 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.804247 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 04:10:56.804354 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:10:56.804572 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":79,"thread_start_us":518,"threads_started":1}
I20260807 04:10:56.806169 30383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7c98abb19048ea95bc30b68c7ffe06" candidate_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110f917e652f4eb8a823716fc2344c02" is_pre_election: true
I20260807 04:10:56.806478 30523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7c98abb19048ea95bc30b68c7ffe06" candidate_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e7809f7496445e1a9792823a6216bdf" is_pre_election: true
I20260807 04:10:56.806860 30383 raft_consensus.cc:2454] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80c4766d875b4a26b6ad2e9b3fae95e5 in term 0.
I20260807 04:10:56.806978 30523 raft_consensus.cc:2454] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80c4766d875b4a26b6ad2e9b3fae95e5 in term 0.
I20260807 04:10:56.807657 30166 leader_election.cc:302] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 110f917e652f4eb8a823716fc2344c02, 7e7809f7496445e1a9792823a6216bdf, 80c4766d875b4a26b6ad2e9b3fae95e5; no voters:
I20260807 04:10:56.808163 30598 raft_consensus.cc:2781] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:10:56.808350 30598 raft_consensus.cc:482] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:10:56.808487 30598 raft_consensus.cc:3037] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:56.810803 30598 raft_consensus.cc:504] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.811420 30598 leader_election.cc:288] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [CANDIDATE]: Term 1 election: Requested vote from peers 110f917e652f4eb8a823716fc2344c02 (127.18.193.2:38767), 7e7809f7496445e1a9792823a6216bdf (127.18.193.3:33187)
I20260807 04:10:56.812012 30383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7c98abb19048ea95bc30b68c7ffe06" candidate_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110f917e652f4eb8a823716fc2344c02"
I20260807 04:10:56.812012 30523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7c98abb19048ea95bc30b68c7ffe06" candidate_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e7809f7496445e1a9792823a6216bdf"
I20260807 04:10:56.812239 30523 raft_consensus.cc:3037] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:56.812239 30383 raft_consensus.cc:3037] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:10:56.815537 30523 raft_consensus.cc:2454] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80c4766d875b4a26b6ad2e9b3fae95e5 in term 1.
I20260807 04:10:56.815537 30383 raft_consensus.cc:2454] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80c4766d875b4a26b6ad2e9b3fae95e5 in term 1.
I20260807 04:10:56.816027 30168 leader_election.cc:302] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 110f917e652f4eb8a823716fc2344c02, 80c4766d875b4a26b6ad2e9b3fae95e5; no voters:
I20260807 04:10:56.816346 30598 raft_consensus.cc:2781] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:10:56.816951 30598 raft_consensus.cc:686] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 1 LEADER]: Becoming Leader. State: Replica: 80c4766d875b4a26b6ad2e9b3fae95e5, State: Running, Role: LEADER
I20260807 04:10:56.817495 30598 consensus_queue.cc:237] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } }
I20260807 04:10:56.822016 30081 catalog_manager.cc:5495] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80c4766d875b4a26b6ad2e9b3fae95e5 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c4766d875b4a26b6ad2e9b3fae95e5" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 46785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 } health_report { overall_health: UNKNOWN } } }
I20260807 04:10:56.865638 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.860352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49244 (local address 127.18.193.1:46785)
0807 04:10:56.860586 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.860594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.860617 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:10:56.860800 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.860804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.860883 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.861005 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:10:56.861382 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.862130 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.863456 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.863868 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.863974 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.864533 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:10:56.865120 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:10:56.865232 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.865347 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.865413 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:10:56.874372 30383 raft_consensus.cc:1261] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 1 FOLLOWER]: Refusing update from remote peer 80c4766d875b4a26b6ad2e9b3fae95e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:56.874372 30523 raft_consensus.cc:1261] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 1 FOLLOWER]: Refusing update from remote peer 80c4766d875b4a26b6ad2e9b3fae95e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:10:56.875700 30601 consensus_queue.cc:1035] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e7809f7496445e1a9792823a6216bdf" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:56.876124 30598 consensus_queue.cc:1035] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "110f917e652f4eb8a823716fc2344c02" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 38767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:10:56.888485 30604 mvcc.cc:205] Tried to move back new op lower bound from 7315766709745680384 to 7315766709527728128. Current Snapshot: MvccSnapshot[applied={T|T < 7315766709745680384}]
I20260807 04:10:56.890597 30606 mvcc.cc:205] Tried to move back new op lower bound from 7315766709745680384 to 7315766709527728128. Current Snapshot: MvccSnapshot[applied={T|T < 7315766709745680384}]
I20260807 04:10:56.901973 30083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:56.902297 30083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:10:56.905952 30083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 516ec7aa20dc4b2d82c7a90923aefe0e: Sending DeleteTablet for 3 replicas of tablet 4b7c98abb19048ea95bc30b68c7ffe06
I20260807 04:10:56.907312 30217 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7c98abb19048ea95bc30b68c7ffe06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:56 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:49242
I20260807 04:10:56.907449 30502 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7c98abb19048ea95bc30b68c7ffe06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:56 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:37100
I20260807 04:10:56.907560 30360 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7c98abb19048ea95bc30b68c7ffe06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:10:56 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:39422
I20260807 04:10:56.909312 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 30149
I20260807 04:10:56.918766 30615 tablet_replica.cc:330] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: stopping tablet replica
I20260807 04:10:56.919353 30615 raft_consensus.cc:2229] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:56.919975 30615 raft_consensus.cc:2258] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:56.920996 30614 tablet_replica.cc:330] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: stopping tablet replica
I20260807 04:10:56.921475 30614 raft_consensus.cc:2229] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:10:56.921976 30614 raft_consensus.cc:2258] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:56.923486 30613 tablet_replica.cc:330] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: stopping tablet replica
I20260807 04:10:56.923889 30613 raft_consensus.cc:2229] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:10:56.924623 30613 raft_consensus.cc:2258] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:10:56.925254 30615 ts_tablet_manager.cc:1905] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:56.925848 30614 ts_tablet_manager.cc:1905] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:56.932844 30615 ts_tablet_manager.cc:1918] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:56.933056 30615 log.cc:1196] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/4b7c98abb19048ea95bc30b68c7ffe06
I20260807 04:10:56.933602 30615 ts_tablet_manager.cc:1939] T 4b7c98abb19048ea95bc30b68c7ffe06 P 110f917e652f4eb8a823716fc2344c02: Deleting consensus metadata
I20260807 04:10:56.935648 30067 catalog_manager.cc:4841] TS 110f917e652f4eb8a823716fc2344c02 (127.18.193.2:38767): tablet 4b7c98abb19048ea95bc30b68c7ffe06 (table test-table [id=3db6fcbdbf3941839edaec6186ea9121]) successfully deleted
Aug 07 04:10:56 dist-test-slave-8n49 krb5kdc[30034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:10:56.939100 30613 ts_tablet_manager.cc:1905] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:10:56.940738 30614 ts_tablet_manager.cc:1918] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:56.940927 30614 log.cc:1196] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/4b7c98abb19048ea95bc30b68c7ffe06
I20260807 04:10:56.941476 30614 ts_tablet_manager.cc:1939] T 4b7c98abb19048ea95bc30b68c7ffe06 P 7e7809f7496445e1a9792823a6216bdf: Deleting consensus metadata
I20260807 04:10:56.943497 30064 catalog_manager.cc:4841] TS 7e7809f7496445e1a9792823a6216bdf (127.18.193.3:33187): tablet 4b7c98abb19048ea95bc30b68c7ffe06 (table test-table [id=3db6fcbdbf3941839edaec6186ea9121]) successfully deleted
I20260807 04:10:56.946378 30613 ts_tablet_manager.cc:1918] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:10:56.946552 30613 log.cc:1196] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/4b7c98abb19048ea95bc30b68c7ffe06
I20260807 04:10:56.947114 30613 ts_tablet_manager.cc:1939] T 4b7c98abb19048ea95bc30b68c7ffe06 P 80c4766d875b4a26b6ad2e9b3fae95e5: Deleting consensus metadata
I20260807 04:10:56.948947 30064 catalog_manager.cc:4841] TS 80c4766d875b4a26b6ad2e9b3fae95e5 (127.18.193.1:46785): tablet 4b7c98abb19048ea95bc30b68c7ffe06 (table test-table [id=3db6fcbdbf3941839edaec6186ea9121]) successfully deleted
I20260807 04:10:56.952402 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:56.910935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.18.193.1:46785)
0807 04:10:56.911160 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:56.911168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:56.922805 (+ 11637us) server_negotiation.cc:407] Connection header received
0807 04:10:56.923014 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:56.923019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:56.923132 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:56.923257 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:56.925595 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.930583 (+ 4988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.932999 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:56.933767 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:56.933908 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:56.939593 (+ 5685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:56.939606 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:56.939627 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:56.939682 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:56.947477 (+ 7795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:56.948542 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:56.948605 (+ 63us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:56.948609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:56.948901 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:56.949548 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:56.949556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:56.949560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:56.950302 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:56.950667 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:56.951748 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0807 04:10:56.952001 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:10:56.953595 30283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:57.068053 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 30149
I20260807 04:10:57.088819 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 30295
Aug 07 04:10:57 dist-test-slave-8n49 krb5kdc[30034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:10:57.104053 30586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:57.089609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.18.193.2:38767)
0807 04:10:57.089815 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:57.089823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:57.089850 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:10:57.090123 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:57.090129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:57.090202 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:57.090295 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:57.091598 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:57.092443 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:57.093826 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:57.094259 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:57.094373 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:57.098489 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:57.098504 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:57.098520 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:57.098560 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:57.101521 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:57.102204 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:57.102212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:57.102214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:57.102356 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:57.102696 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:57.102700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:57.102702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:57.103123 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:57.103343 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:57.103605 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 04:10:57.103770 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 04:10:57.104805 30425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:57.195443 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 30295
I20260807 04:10:57.214378 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 30436
Aug 07 04:10:57 dist-test-slave-8n49 krb5kdc[30034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:10:57.231168 30588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:10:57.215167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.18.193.3:33187)
0807 04:10:57.215336 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 04:10:57.215340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:10:57.215359 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:10:57.215462 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:10:57.215465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:10:57.215564 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:10:57.215652 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:10:57.217056 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:57.217830 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:57.219416 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:10:57.219798 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:10:57.219892 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:10:57.225458 (+ 5566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:10:57.225475 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:10:57.225491 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:10:57.225526 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:10:57.228179 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:57.229015 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:57.229022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:57.229025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:57.229183 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:10:57.229600 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:10:57.229605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:10:57.229607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:10:57.229980 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:10:57.230288 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 04:10:57.230598 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 04:10:57.230816 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:10:57.231803 30566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:57.323350 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 30436
I20260807 04:10:57.341742 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 30047
I20260807 04:10:57.343042 30114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:10:57.458528 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 30047
2026-08-07T04:10:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4897 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:10:57 dist-test-slave-8n49 krb5kdc[30623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:10:57 dist-test-slave-8n49 krb5kdc[30623](info): set up 2 sockets
Aug 07 04:10:57 dist-test-slave-8n49 krb5kdc[30623](info): commencing operation
krb5kdc: starting...
W20260807 04:10:58.615536 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:10:58 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:10:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/rangeradmin_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/rangeradmin_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/rangerlookup_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/rangerlookup_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
I20260807 04:10:59.011557 19204 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5Tobyc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/postgres -l logfile start
2026-08-07 04:11:01.820 UTC [30655] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 04:11:01.820 UTC [30655] LOG: listening on IPv4 address "127.18.193.20", port 53075
2026-08-07 04:11:01.825 UTC [30655] LOG: listening on Unix socket "/tmp/.s.PGSQL.53075"
2026-08-07 04:11:01.850 UTC [30658] LOG: database system was shut down at 2026-08-07 04:11:00 UTC
2026-08-07 04:11:01.854 UTC [30655] LOG: database system is ready to accept connections
I20260807 04:11:02.874845 19204 mini_postgres.cc:96] Postgres bound to 53075
2026-08-07 04:11:02.899 UTC [30666] FATAL: database "slave" does not exist
127.18.193.20:53075 - accepting connections
I20260807 04:11:02.900146 19204 mini_ranger.cc:161] Starting Ranger...
I20260807 04:11:02.935887 19204 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 04:11:03.534794 19204 mini_ranger.cc:87] Created ranger Postgres database
I20260807 04:11:03.534946 19204 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-admin
2026-08-07 04:11:03,870 [I] DB FLAVOR :POSTGRES
2026-08-07 04:11:03,871 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:11:03,871 [I] Checking connection..
2026-08-07 04:11:03,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 04:11:04,116 [I] Checking connection passed.
2026-08-07 04:11:04,116 [I] --------- Verifying version history table ---------
2026-08-07 04:11:04,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:11:04,376 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:11:04,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:11:04,635 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:11:04,635 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 04:11:04,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 04:11:04.899 UTC [30734] WARNING: there is no transaction in progress
2026-08-07 04:11:04,906 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 04:11:04,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:11:05,161 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 04:11:05,161 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:11:05,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 04:11:05,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 04:11:05,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8n49.c.gcp-upstream.internal','N') ;"
2026-08-07 04:11:05,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:11:06,168 [I] Table x_portal_user does not exist in database ranger
2026-08-07 04:11:06,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:11:06,433 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 04:11:06,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:11:06,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:11:06,936 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 04:11:06,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 04:11:08.011 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.030 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.048 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.065 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.084 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.104 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.168 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.175 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.190 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.201 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.212 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.227 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.237 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.245 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.253 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08.260 UTC [30847] WARNING: there is no transaction in progress
2026-08-07 04:11:08,974 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 04:11:08,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 04:11:09,205 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 04:11:09,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:11:09,476 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 04:11:09,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:11:09,741 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 04:11:09,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:11:09,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:11:10,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8n49.c.gcp-upstream.internal';"
2026-08-07 04:11:10,467 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 04:11:10,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 04:11:10,702 [I] DB_PATCHES have already been applied
I20260807 04:11:10.709057 19204 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-admin:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5Tobyc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 04:11:10.709156 19204 mini_ranger.cc:193] Using host: 127.18.193.20
I20260807 04:11:10.729503 19204 mini_ranger.cc:227] Ranger admin URL: http://127.18.193.20:43669
Aug 07, 2026 4:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 4:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 4:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 4:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 4:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 04:11:11.912930 19204 mini_ranger.cc:160] Time spent starting Ranger: real 9.013s user 0.008s sys 0.054s
Aug 07, 2026 4:11:33 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 4:11:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 4:11:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 4:11:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 4:11:34 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 04:11:34.722950 19204 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/rangerkms_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/rangerkms_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.193.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.193.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 04:11:34 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075894, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 04:11:34.946719 19204 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 04:11:34.974574 19204 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 04:11:36.025614 19204 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 04:11:36.025801 19204 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms
I20260807 04:11:36.025949 19204 mini_ranger_kms.cc:227] Using postgres at 127.18.193.20:53075
2026-08-07 04:11:37,267 [I] DB FLAVOR :POSTGRES
2026-08-07 04:11:37,268 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:11:37,268 [I] Checking connection
2026-08-07 04:11:37,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 04:11:37,606 [I] connection success
2026-08-07 04:11:37,606 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 04:11:37,607 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 04:11:37,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 04:11:37,905 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 04:11:37,905 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:11:37,905 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 04:11:37,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:53075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 04:11:38,201 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 04:11:38.727165 19204 mini_ranger_kms.cc:310] Created kms service
I20260807 04:11:38.956226 19204 mini_ranger_kms.cc:326] Created kudu user
I20260807 04:11:39.005463 19204 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 04:11:39 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075894, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:11:39.446162 19204 mini_ranger_kms.cc:384] Added ranger policy
I20260807 04:11:39.447758 19204 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms:/tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 04:11:39.447837 19204 mini_ranger_kms.cc:240] Using host: 127.18.193.20
I20260807 04:11:39.474857 19204 mini_ranger_kms.cc:276] Ranger KMS PID: 31042
I20260807 04:11:39.475099 19204 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.193.20:51167
Aug 07, 2026 4:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 4:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 4:11:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 4:11:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 04:11:41.933354 19204 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.987s user 0.179s sys 0.507s
I20260807 04:11:41.933724 19204 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 04:11:41.933934 19204 mini_ranger_kms.cc:401] 127.18.193.20:51167/kms/v1/keys
Aug 07 04:11:42 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786075902, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.193.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 04:11:42 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075902, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.193.20@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
Aug 07 04:11:42 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075902, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.193.20@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
Aug 07 04:11:45 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:11:45.478018 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41773
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:41773
--ranger_config_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:51167/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:11:45.993520 31097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:11:45.993825 31097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:11:45.993986 31097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:11:46.002913 31097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:11:46.003116 31097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:11:46.003264 31097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:11:46.003460 31097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:11:46.003604 31097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:11:46.016307 31097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.193.20:51167/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:41773
--ranger_config_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:11:46.017884 31097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:11:46.019384 31097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:11:46.034768 31103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:46.034803 31102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:46.035508 31105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:11:46.035801 31097 server_base.cc:1034] running on GCE node
I20260807 04:11:46.036680 31097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:11:46.038093 31097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:11:46.039304 31097 hybrid_clock.cc:648] HybridClock initialized: now 1786075906039258 us; error 34 us; skew 500 ppm
I20260807 04:11:46.039747 31097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:11:46 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:11:46.046425 31097 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:11:46.048316 31097 webserver.cc:460] Webserver started at http://127.18.193.62:34765/ using document root <none> and password file <none>
I20260807 04:11:46.049279 31097 fs_manager.cc:359] Metadata directory not provided
I20260807 04:11:46.049455 31097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:11:46.049787 31097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:11:46 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:11:46.114167 31097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "1ac2f966457049b68788471344dc0725"
format_stamp: "Formatted at 2026-08-07 04:11:46 on dist-test-slave-8n49"
server_key: "068c73f9601e1c7c107c041b3275701e"
server_key_iv: "8e8edb2c2f5328dff22c5015acd0ed19"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:46.115288 31097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "1ac2f966457049b68788471344dc0725"
format_stamp: "Formatted at 2026-08-07 04:11:46 on dist-test-slave-8n49"
server_key: "068c73f9601e1c7c107c041b3275701e"
server_key_iv: "8e8edb2c2f5328dff22c5015acd0ed19"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:46.121369 31097 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 04:11:46.139360 31112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:46.141137 31097 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 04:11:46.141321 31097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "1ac2f966457049b68788471344dc0725"
format_stamp: "Formatted at 2026-08-07 04:11:46 on dist-test-slave-8n49"
server_key: "068c73f9601e1c7c107c041b3275701e"
server_key_iv: "8e8edb2c2f5328dff22c5015acd0ed19"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:46.141559 31097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:11:46.169879 31097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:11:46.175395 31097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:11:46.175915 31097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:11:46.209293 31097 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:41773
I20260807 04:11:46.209345 31166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:41773 every 8 connection(s)
I20260807 04:11:46.211314 31097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:11:46.216362 31167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:11:46.221498 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 31097
I20260807 04:11:46.221732 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:11:46.222556 19204 external_mini_cluster.cc:1428] Setting key 2ca659d34a3436563a562e31185f5a34
I20260807 04:11:46.231387 31167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725: Bootstrap starting.
I20260807 04:11:46.235929 31167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725: Neither blocks nor log segments found. Creating new log.
I20260807 04:11:46.238163 31167 log.cc:824] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:11:46 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:11:46.242239 31167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725: No bootstrap required, opened a new log
I20260807 04:11:46.250797 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:46.225127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.18.193.62:41773)
0807 04:11:46.226970 (+ 1843us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:46.226983 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:46.227048 (+ 65us) server_negotiation.cc:407] Connection header received
0807 04:11:46.229265 (+ 2217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:46.229290 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:46.229722 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:46.230075 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:46.231918 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:46.233419 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:46.235353 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:46.236159 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:46.236289 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:46.241710 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:46.241719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:46.241732 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:46.241770 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:46.245908 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:46.246719 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:46.246740 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:46.246747 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:46.246964 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:46.247427 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:46.247433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:46.247436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:46.248095 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:46.248576 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:46.249095 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 04:11:46.249516 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1593,"thread_start_us":1223,"threads_started":1}
I20260807 04:11:46.254657 31167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac2f966457049b68788471344dc0725" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41773 } }
I20260807 04:11:46.255330 31167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:11:46.255494 31167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac2f966457049b68788471344dc0725, State: Initialized, Role: FOLLOWER
I20260807 04:11:46.256265 31167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac2f966457049b68788471344dc0725" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41773 } }
I20260807 04:11:46.256664 31167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:11:46.256880 31167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:11:46.257143 31167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:11:46.260658 31167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac2f966457049b68788471344dc0725" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41773 } }
I20260807 04:11:46.261349 31167 leader_election.cc:302] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ac2f966457049b68788471344dc0725; no voters:
I20260807 04:11:46.262727 31167 leader_election.cc:288] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:11:46.262893 31172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:11:46.266208 31172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [term 1 LEADER]: Becoming Leader. State: Replica: 1ac2f966457049b68788471344dc0725, State: Running, Role: LEADER
I20260807 04:11:46.267333 31167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:11:46.267148 31172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac2f966457049b68788471344dc0725" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41773 } }
I20260807 04:11:46.271832 31173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ac2f966457049b68788471344dc0725" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac2f966457049b68788471344dc0725" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41773 } } }
I20260807 04:11:46.272607 31173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [sys.catalog]: This master's current role is: LEADER
I20260807 04:11:46.274093 31174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ac2f966457049b68788471344dc0725. Latest consensus state: current_term: 1 leader_uuid: "1ac2f966457049b68788471344dc0725" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac2f966457049b68788471344dc0725" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41773 } } }
I20260807 04:11:46.274955 31174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725 [sys.catalog]: This master's current role is: LEADER
I20260807 04:11:46.278234 31179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:11:46.288897 31179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:11:46.290952 31167 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 04:11:46.314949 31179 catalog_manager.cc:1347] Generated new cluster ID: 5ed2cadecdd74b86a9be14e856001e63
I20260807 04:11:46.315128 31179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:11:46.332633 31179 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:11:46.334896 31179 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:11:46.344901 31179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725: Generated new TSK 0
I20260807 04:11:46.346231 31179 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:11:47.222170 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:51167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41773
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:11:47 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786075907, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 04:11:47.822638 31214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:11:47.823424 31214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:11:47.823688 31214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:11:47.837431 31214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:11:47.837697 31214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:11:47.837946 31214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:11:47.859665 31214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.193.20:51167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:11:47.862035 31214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:11:47.864184 31214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:11:47.891137 31226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:47.891321 31228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:47.891613 31225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:11:47.893633 31214 server_base.cc:1034] running on GCE node
I20260807 04:11:47.894418 31214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:11:47.917290 31214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:11:47.919004 31214 hybrid_clock.cc:648] HybridClock initialized: now 1786075907918885 us; error 88 us; skew 500 ppm
I20260807 04:11:47.919612 31214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:11:47 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075907, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:11:47.924217 31214 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:11:47.928766 31214 webserver.cc:460] Webserver started at http://127.18.193.1:45367/ using document root <none> and password file <none>
I20260807 04:11:47.930001 31214 fs_manager.cc:359] Metadata directory not provided
I20260807 04:11:47.930279 31214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:11:47.930819 31214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:11:47 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075907, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:11:47.955583 31214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a"
format_stamp: "Formatted at 2026-08-07 04:11:47 on dist-test-slave-8n49"
server_key: "a0231134e53e68a57392103762f4d733"
server_key_iv: "ab76f9773181ff0230cba206f22d2596"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:47.957645 31214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a"
format_stamp: "Formatted at 2026-08-07 04:11:47 on dist-test-slave-8n49"
server_key: "a0231134e53e68a57392103762f4d733"
server_key_iv: "ab76f9773181ff0230cba206f22d2596"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:47.980969 31214 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.009s sys 0.012s
I20260807 04:11:48.029261 31236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:48.031368 31214 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.004s sys 0.002s
I20260807 04:11:48.031683 31214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a"
format_stamp: "Formatted at 2026-08-07 04:11:47 on dist-test-slave-8n49"
server_key: "a0231134e53e68a57392103762f4d733"
server_key_iv: "ab76f9773181ff0230cba206f22d2596"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:48.032050 31214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:11:48.061293 31214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:11:48.070560 31214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:11:48.071264 31214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:11:48.076893 31214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 07 04:11:48 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075907, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:11:48.120661 31214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:11:48.120955 31214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 04:11:48.121239 31214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:11:48.121417 31214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:11:48 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075907, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:11:48.629567 31214 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:38063
I20260807 04:11:48.629987 31355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:38063 every 8 connection(s)
I20260807 04:11:48.631426 31214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:11:48.639871 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 31214
I20260807 04:11:48.640133 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:11:48.641199 19204 external_mini_cluster.cc:1428] Setting key 8a093b1ecf14428f59b83a1d48defd19
Aug 07 04:11:48 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075907, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:11:48.672984 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:48.637418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:41777 (local address 127.18.193.62:41773)
0807 04:11:48.638388 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:48.638398 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:48.641084 (+ 2686us) server_negotiation.cc:407] Connection header received
0807 04:11:48.645212 (+ 4128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:48.645220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:48.645316 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:48.645403 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:48.648675 (+ 3272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:48.650148 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:48.653588 (+ 3440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:48.654700 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:48.654837 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:48.662338 (+ 7501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:48.662352 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:48.662361 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:48.662413 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:48.667444 (+ 5031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:48.668638 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:48.668645 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:48.668649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:48.668854 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:48.669562 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:48.669571 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:48.669578 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:48.670096 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:48.670242 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:48.672029 (+ 1787us) server_negotiation.cc:299] Negotiation successful
0807 04:11:48.672339 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":521,"threads_started":1}
I20260807 04:11:48.675125 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:48.638342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41773 (local address 127.18.193.1:41777)
0807 04:11:48.640816 (+ 2474us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:48.640879 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:48.644815 (+ 3936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:48.646166 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:48.646180 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:48.647087 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:11:48.648384 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:48.648419 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:48.650396 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:48.650405 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:48.653300 (+ 2895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:48.653395 (+ 95us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:48.655481 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:48.655495 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:48.655972 (+ 477us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:48.657098 (+ 1126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:11:48.657124 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:11:48.661911 (+ 4787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:11:48.667672 (+ 5761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:11:48.667681 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:11:48.667696 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:11:48.668446 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:11:48.669008 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:11:48.669016 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:11:48.669022 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:11:48.669413 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:11:48.670921 (+ 1508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:11:48.670932 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:11:48.671517 (+ 585us) client_negotiation.cc:769] Sending connection context
0807 04:11:48.671952 (+ 435us) client_negotiation.cc:240] Negotiation successful
0807 04:11:48.673390 (+ 1438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1922,"thread_start_us":3580,"threads_started":1}
I20260807 04:11:48.677459 31356 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41773
I20260807 04:11:48.677848 31356 heartbeater.cc:464] Registering TS with master...
I20260807 04:11:48.679114 31356 heartbeater.cc:511] Master 127.18.193.62:41773 requested a full tablet report, sending...
I20260807 04:11:48.682420 31130 ts_manager.cc:194] Registered new tserver with Master: b1ec2ddbc02e46b0bb95ee190f3d7d1a (127.18.193.1:38063)
I20260807 04:11:48.685304 31130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:41777
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:11:48.735971 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:51167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41773
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:11:49.409121 31364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:11:49.409526 31364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:11:49.409794 31364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:11:49.420279 31364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:11:49.420459 31364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:11:49.420748 31364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:11:49.433552 31364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.193.20:51167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:11:49.435145 31364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:11:49.436936 31364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:11:49.456727 31377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:49.456766 31378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:49.457060 31380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:11:49.459026 31364 server_base.cc:1034] running on GCE node
I20260807 04:11:49.459830 31364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:11:49.461254 31364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:11:49.462572 31364 hybrid_clock.cc:648] HybridClock initialized: now 1786075909462421 us; error 89 us; skew 500 ppm
I20260807 04:11:49.463132 31364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:11:49 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075909, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:11:49.466928 31364 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:11:49.468861 31364 webserver.cc:460] Webserver started at http://127.18.193.2:42165/ using document root <none> and password file <none>
I20260807 04:11:49.469650 31364 fs_manager.cc:359] Metadata directory not provided
I20260807 04:11:49.469810 31364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:11:49.470152 31364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:11:49 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075909, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:11:49.491185 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "c4300709c5b7445b8dd9063185d945e0"
format_stamp: "Formatted at 2026-08-07 04:11:49 on dist-test-slave-8n49"
server_key: "09c5c68befe37cb8d35f88787aea17c4"
server_key_iv: "55b54d4c61e396621502019a410f02a9"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:49.492419 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "c4300709c5b7445b8dd9063185d945e0"
format_stamp: "Formatted at 2026-08-07 04:11:49 on dist-test-slave-8n49"
server_key: "09c5c68befe37cb8d35f88787aea17c4"
server_key_iv: "55b54d4c61e396621502019a410f02a9"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:49.498839 31364 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 04:11:49.518146 31387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:49.519790 31364 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260807 04:11:49.520006 31364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "c4300709c5b7445b8dd9063185d945e0"
format_stamp: "Formatted at 2026-08-07 04:11:49 on dist-test-slave-8n49"
server_key: "09c5c68befe37cb8d35f88787aea17c4"
server_key_iv: "55b54d4c61e396621502019a410f02a9"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:49.520267 31364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:11:49.533893 31364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:11:49.539119 31364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:11:49.539556 31364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:11:49.540997 31364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:11:49.542866 31364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:11:49.542965 31364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:49.543179 31364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:11:49.543296 31364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:49.607710 31364 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:45317
I20260807 04:11:49.607872 31504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:45317 every 8 connection(s)
I20260807 04:11:49.609295 31364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:11:49.615257 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 31364
I20260807 04:11:49.615456 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:11:49.615947 19204 external_mini_cluster.cc:1428] Setting key 23efeca1c5c95692f975a25250c03dee
Aug 07 04:11:49 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075909, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:11:49.639045 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:49.612237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:38217 (local address 127.18.193.62:41773)
0807 04:11:49.613065 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:49.613072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:49.613814 (+ 742us) server_negotiation.cc:407] Connection header received
0807 04:11:49.615815 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:49.615821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:49.615911 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:49.616048 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:49.618078 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:49.619073 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:49.622423 (+ 3350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:49.622865 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:49.622969 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:49.628615 (+ 5646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:49.628626 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:49.628634 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:49.628688 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:49.633626 (+ 4938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:49.635225 (+ 1599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:49.635234 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:49.635237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:49.635432 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:49.636181 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:49.636192 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:49.636195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:49.636734 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:49.636940 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:49.638024 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0807 04:11:49.638315 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":522,"threads_started":1}
I20260807 04:11:49.640151 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:49.612816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41773 (local address 127.18.193.2:38217)
0807 04:11:49.613651 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:49.613697 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:49.615500 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:49.616386 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:49.616395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:49.616935 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:11:49.617877 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:49.617906 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:49.619265 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:49.619271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:49.622201 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:49.622255 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:49.623184 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:49.623194 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:49.623670 (+ 476us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:49.624726 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:11:49.624765 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:11:49.628272 (+ 3507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:11:49.633833 (+ 5561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:11:49.633845 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:11:49.633866 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:11:49.634934 (+ 1068us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:11:49.635598 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:11:49.635604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:11:49.635608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:11:49.635933 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:11:49.636908 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:11:49.636917 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:11:49.637404 (+ 487us) client_negotiation.cc:769] Sending connection context
0807 04:11:49.637794 (+ 390us) client_negotiation.cc:240] Negotiation successful
0807 04:11:49.638466 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":334,"threads_started":1}
I20260807 04:11:49.642372 31505 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41773
I20260807 04:11:49.642900 31505 heartbeater.cc:464] Registering TS with master...
I20260807 04:11:49.644078 31505 heartbeater.cc:511] Master 127.18.193.62:41773 requested a full tablet report, sending...
I20260807 04:11:49.646308 31130 ts_manager.cc:194] Registered new tserver with Master: c4300709c5b7445b8dd9063185d945e0 (127.18.193.2:45317)
I20260807 04:11:49.647491 31130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:38217
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:11:49.689298 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:51167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41773
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:11:49.690066 31356 heartbeater.cc:503] Master 127.18.193.62:41773 was elected leader, sending a full tablet report...
W20260807 04:11:50.192449 31511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:11:50.192950 31511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:11:50.193171 31511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:11:50.201927 31511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:11:50.202126 31511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:11:50.202347 31511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:11:50.219882 31511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.193.20:51167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:11:50.221908 31511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:11:50.223747 31511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:11:50.240381 31516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:50.240713 31519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:11:50.241448 31517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:11:50.242841 31511 server_base.cc:1034] running on GCE node
I20260807 04:11:50.243482 31511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:11:50.244541 31511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:11:50.245852 31511 hybrid_clock.cc:648] HybridClock initialized: now 1786075910245760 us; error 54 us; skew 500 ppm
I20260807 04:11:50.246263 31511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:11:50 dist-test-slave-8n49 krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075910, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:11:50.250458 31511 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:11:50.252538 31511 webserver.cc:460] Webserver started at http://127.18.193.3:44425/ using document root <none> and password file <none>
I20260807 04:11:50.253676 31511 fs_manager.cc:359] Metadata directory not provided
I20260807 04:11:50.253901 31511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:11:50.254316 31511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:11:50 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075910, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:11:50.270924 31511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "bd843eaf9ad84f73a6ee28a9d1340e72"
format_stamp: "Formatted at 2026-08-07 04:11:50 on dist-test-slave-8n49"
server_key: "c9418f43d64ea7a5348ca5a14c0f551a"
server_key_iv: "18f5758f2c4490fd3b9f105b89f4124b"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:50.271997 31511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "bd843eaf9ad84f73a6ee28a9d1340e72"
format_stamp: "Formatted at 2026-08-07 04:11:50 on dist-test-slave-8n49"
server_key: "c9418f43d64ea7a5348ca5a14c0f551a"
server_key_iv: "18f5758f2c4490fd3b9f105b89f4124b"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:50.279506 31511 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 04:11:50.298300 31526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:50.299942 31511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260807 04:11:50.300218 31511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "bd843eaf9ad84f73a6ee28a9d1340e72"
format_stamp: "Formatted at 2026-08-07 04:11:50 on dist-test-slave-8n49"
server_key: "c9418f43d64ea7a5348ca5a14c0f551a"
server_key_iv: "18f5758f2c4490fd3b9f105b89f4124b"
server_key_version: "kuduclusterkey@0"
I20260807 04:11:50.300523 31511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:11:50.334033 31511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:11:50.340018 31511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:11:50.340631 31511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:11:50.342360 31511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:11:50.344440 31511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:11:50.344655 31511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:50.344904 31511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:11:50.345037 31511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:11:50.409904 31511 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:44489
I20260807 04:11:50.409991 31646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:44489 every 8 connection(s)
I20260807 04:11:50.411541 31511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:11:50.414546 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 31511
I20260807 04:11:50.414743 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:11:50.415481 19204 external_mini_cluster.cc:1428] Setting key e36ba569fc648d8f1ea68f8b66257f30
Aug 07 04:11:50 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075910, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:11:50.437661 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.415000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:56935 (local address 127.18.193.62:41773)
0807 04:11:50.415923 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.415930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.417393 (+ 1463us) server_negotiation.cc:407] Connection header received
0807 04:11:50.420155 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.420160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.420222 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.420312 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:50.422416 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.423332 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.425575 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.426169 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.426276 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.430604 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:50.430609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:50.430612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:50.430678 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:50.434537 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.435250 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.435254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.435257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.435410 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.435861 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.435864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.435866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.436211 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:50.436322 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.437155 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.437356 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":488,"threads_started":1}
I20260807 04:11:50.439016 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.415988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41773 (local address 127.18.193.3:56935)
0807 04:11:50.417216 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:50.417269 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:50.419916 (+ 2647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:50.420708 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:50.420716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:50.421366 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:11:50.422189 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.422209 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.423526 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.423531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.425421 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.425435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.426475 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.426481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.426779 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.427642 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:11:50.427665 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:11:50.430357 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:11:50.434671 (+ 4314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:11:50.434678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:11:50.434692 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:11:50.435128 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:11:50.435531 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:11:50.435534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:11:50.435537 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:11:50.435758 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:11:50.436326 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:11:50.436332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:11:50.436729 (+ 397us) client_negotiation.cc:769] Sending connection context
0807 04:11:50.437056 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 04:11:50.437536 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"thread_start_us":560,"threads_started":1}
I20260807 04:11:50.441210 31647 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41773
I20260807 04:11:50.441583 31647 heartbeater.cc:464] Registering TS with master...
I20260807 04:11:50.442530 31647 heartbeater.cc:511] Master 127.18.193.62:41773 requested a full tablet report, sending...
I20260807 04:11:50.444905 31130 ts_manager.cc:194] Registered new tserver with Master: bd843eaf9ad84f73a6ee28a9d1340e72 (127.18.193.3:44489)
I20260807 04:11:50.446537 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:11:50.446694 31130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:56935
I20260807 04:11:50.470749 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.454006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.18.193.62:41773)
0807 04:11:50.454194 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.454237 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.454890 (+ 653us) server_negotiation.cc:407] Connection header received
0807 04:11:50.455158 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.455162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.455277 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.455445 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:50.457114 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.458103 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.459872 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.460467 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.460632 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.463646 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:50.463651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:50.463655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:50.463689 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:50.467477 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.468229 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.468235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.468239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.468484 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.468930 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.468952 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.468956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.469305 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:50.469658 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.469999 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.470296 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:11:50.475895 31130 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:11:50.492273 31130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:11:50.522370 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.507707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:44489 (local address 127.0.0.1:47002)
0807 04:11:50.510277 (+ 2570us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:50.510338 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:50.510656 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:50.511909 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:50.511916 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:50.511946 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:11:50.512437 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.512458 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.515904 (+ 3446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.515913 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.518034 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.518050 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.520711 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.520719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.520987 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.521121 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 04:11:50.521270 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 04:11:50.521443 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2171,"mutex_wait_us":53,"thread_start_us":3194,"threads_started":1}
I20260807 04:11:50.522440 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.507806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.18.193.3:44489)
0807 04:11:50.508631 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.508643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.510509 (+ 1866us) server_negotiation.cc:407] Connection header received
0807 04:11:50.510910 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.510917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.511275 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.511498 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:11:50.512742 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.515773 (+ 3031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.518321 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.520027 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.520176 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.520460 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.521719 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.521938 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":1101,"threads_started":1}
I20260807 04:11:50.525826 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.508834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.18.193.1:38063)
0807 04:11:50.509622 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.509631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.511794 (+ 2163us) server_negotiation.cc:407] Connection header received
0807 04:11:50.511904 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.511915 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.512342 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.512726 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:11:50.514717 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.517819 (+ 3102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.521296 (+ 3477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.523409 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.523547 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.523894 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.525050 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.525254 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":344,"threads_started":1}
I20260807 04:11:50.525983 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.508443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38063 (local address 127.0.0.1:41272)
0807 04:11:50.511328 (+ 2885us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:50.511374 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:50.511659 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:50.513207 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:50.513214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:50.513239 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:11:50.514401 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.514421 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.518052 (+ 3631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.518058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.521092 (+ 3034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.521106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.524379 (+ 3273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.524386 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.524827 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.524869 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 04:11:50.524999 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 04:11:50.525149 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2722,"mutex_wait_us":30,"thread_start_us":1138,"threads_started":1}
I20260807 04:11:50.528071 31575 tablet_service.cc:1467] Processing CreateTablet for tablet baac0e0b11224ca1899c09ed6749d72b (DEFAULT_TABLE table=test-table [id=71f655ff0a8b4121bf0ff510272fda7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:11:50.530300 31575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baac0e0b11224ca1899c09ed6749d72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:11:50.529958 31287 tablet_service.cc:1467] Processing CreateTablet for tablet baac0e0b11224ca1899c09ed6749d72b (DEFAULT_TABLE table=test-table [id=71f655ff0a8b4121bf0ff510272fda7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:11:50.531430 31287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baac0e0b11224ca1899c09ed6749d72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:11:50.533010 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.511756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47314 (local address 127.18.193.2:45317)
0807 04:11:50.518706 (+ 6950us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.518714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.519790 (+ 1076us) server_negotiation.cc:407] Connection header received
0807 04:11:50.520079 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.520088 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.520354 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.520636 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:11:50.522312 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.524374 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.529069 (+ 4695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.530928 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.531095 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.531287 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.532304 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.532543 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6805,"thread_start_us":6695,"threads_started":1}
I20260807 04:11:50.533332 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.513368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45317 (local address 127.0.0.1:47314)
0807 04:11:50.519578 (+ 6210us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:50.519600 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:50.519828 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:50.520969 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:50.520974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:50.521011 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:11:50.521913 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.521928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.524826 (+ 2898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.524832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.528818 (+ 3986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.528839 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.531778 (+ 2939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.531788 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.532042 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.532102 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 04:11:50.532217 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 04:11:50.532386 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6078,"mutex_wait_us":1,"thread_start_us":5881,"threads_started":1}
I20260807 04:11:50.538712 31436 tablet_service.cc:1467] Processing CreateTablet for tablet baac0e0b11224ca1899c09ed6749d72b (DEFAULT_TABLE table=test-table [id=71f655ff0a8b4121bf0ff510272fda7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:11:50.540582 31436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baac0e0b11224ca1899c09ed6749d72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:11:50.548344 31667 tablet_bootstrap.cc:492] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Bootstrap starting.
I20260807 04:11:50.554093 31667 tablet_bootstrap.cc:654] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Neither blocks nor log segments found. Creating new log.
I20260807 04:11:50.555891 31668 tablet_bootstrap.cc:492] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Bootstrap starting.
I20260807 04:11:50.556625 31667 log.cc:824] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:11:50.561590 31668 tablet_bootstrap.cc:654] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Neither blocks nor log segments found. Creating new log.
I20260807 04:11:50.561900 31667 tablet_bootstrap.cc:492] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: No bootstrap required, opened a new log
I20260807 04:11:50.562286 31670 tablet_bootstrap.cc:492] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Bootstrap starting.
I20260807 04:11:50.562480 31667 ts_tablet_manager.cc:1397] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260807 04:11:50.564412 31668 log.cc:824] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Log is configured to *not* fsync() on all Append() calls
I20260807 04:11:50.566725 31670 tablet_bootstrap.cc:654] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Neither blocks nor log segments found. Creating new log.
I20260807 04:11:50.568200 31670 log.cc:824] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:11:50.568706 31668 tablet_bootstrap.cc:492] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: No bootstrap required, opened a new log
I20260807 04:11:50.569015 31668 ts_tablet_manager.cc:1397] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260807 04:11:50.571372 31670 tablet_bootstrap.cc:492] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: No bootstrap required, opened a new log
I20260807 04:11:50.571820 31670 ts_tablet_manager.cc:1397] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260807 04:11:50.574817 31667 raft_consensus.cc:348] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.575505 31667 raft_consensus.cc:374] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:11:50.575682 31667 raft_consensus.cc:729] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ec2ddbc02e46b0bb95ee190f3d7d1a, State: Initialized, Role: FOLLOWER
I20260807 04:11:50.576586 31667 consensus_queue.cc:260] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.579284 31667 ts_tablet_manager.cc:1428] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260807 04:11:50.581678 31670 raft_consensus.cc:348] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.582098 31670 raft_consensus.cc:374] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:11:50.582240 31670 raft_consensus.cc:729] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4300709c5b7445b8dd9063185d945e0, State: Initialized, Role: FOLLOWER
I20260807 04:11:50.582090 31668 raft_consensus.cc:348] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.582553 31668 raft_consensus.cc:374] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:11:50.582666 31668 raft_consensus.cc:729] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd843eaf9ad84f73a6ee28a9d1340e72, State: Initialized, Role: FOLLOWER
I20260807 04:11:50.582906 31670 consensus_queue.cc:260] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.583267 31668 consensus_queue.cc:260] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.584182 31505 heartbeater.cc:503] Master 127.18.193.62:41773 was elected leader, sending a full tablet report...
I20260807 04:11:50.584895 31670 ts_tablet_manager.cc:1428] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260807 04:11:50.585784 31647 heartbeater.cc:503] Master 127.18.193.62:41773 was elected leader, sending a full tablet report...
I20260807 04:11:50.586510 31668 ts_tablet_manager.cc:1428] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Time spent starting tablet: real 0.017s user 0.009s sys 0.008s
I20260807 04:11:50.596151 31673 raft_consensus.cc:482] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:11:50.596632 31673 raft_consensus.cc:504] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.598721 31673 leader_election.cc:288] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd843eaf9ad84f73a6ee28a9d1340e72 (127.18.193.3:44489), c4300709c5b7445b8dd9063185d945e0 (127.18.193.2:45317)
I20260807 04:11:50.607024 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.599128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:41945 (local address 127.18.193.3:44489)
0807 04:11:50.599352 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.599357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.599971 (+ 614us) server_negotiation.cc:407] Connection header received
0807 04:11:50.600332 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.600342 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.600436 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.600518 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:11:50.601397 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.602224 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.604451 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.605699 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.605931 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.606046 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.606555 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.606693 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:11:50.607226 31676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.599063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:44489 (local address 127.18.193.1:41945)
0807 04:11:50.599858 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:50.599877 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:50.600089 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:50.600782 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:50.600785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:50.600808 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:11:50.601242 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.601251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.602364 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.602372 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.604278 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.604293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.606017 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.606024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.606235 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.606291 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 04:11:50.606510 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 04:11:50.606696 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":368,"threads_started":1}
I20260807 04:11:50.607678 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.600216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:41863 (local address 127.18.193.2:45317)
0807 04:11:50.600523 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.600529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.600865 (+ 336us) server_negotiation.cc:407] Connection header received
0807 04:11:50.601091 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.601097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.601176 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.601274 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:11:50.601935 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.602689 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.604990 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.606143 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.606311 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.606404 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.606829 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.606951 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260807 04:11:50.608269 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.599877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45317 (local address 127.18.193.1:41863)
0807 04:11:50.600755 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0807 04:11:50.600769 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:11:50.600946 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:11:50.601288 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:11:50.601292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:11:50.601316 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:11:50.601785 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.601794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.602835 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.602840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.604834 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:11:50.604850 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.606430 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.606437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:11:50.606645 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.606677 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:11:50.607513 (+ 836us) client_negotiation.cc:240] Negotiation successful
0807 04:11:50.607665 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":504,"threads_started":1}
I20260807 04:11:50.608352 31596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baac0e0b11224ca1899c09ed6749d72b" candidate_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" is_pre_election: true
I20260807 04:11:50.609316 31596 raft_consensus.cc:2454] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ec2ddbc02e46b0bb95ee190f3d7d1a in term 0.
I20260807 04:11:50.610242 31238 leader_election.cc:302] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1ec2ddbc02e46b0bb95ee190f3d7d1a, bd843eaf9ad84f73a6ee28a9d1340e72; no voters:
I20260807 04:11:50.610227 31457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baac0e0b11224ca1899c09ed6749d72b" candidate_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4300709c5b7445b8dd9063185d945e0" is_pre_election: true
I20260807 04:11:50.610759 31673 raft_consensus.cc:2781] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:11:50.611012 31673 raft_consensus.cc:482] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:11:50.611199 31673 raft_consensus.cc:3037] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:11:50.611202 31457 raft_consensus.cc:2454] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ec2ddbc02e46b0bb95ee190f3d7d1a in term 0.
W20260807 04:11:50.612108 31506 tablet.cc:2369] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:11:50.615269 31673 raft_consensus.cc:504] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.616282 31673 leader_election.cc:288] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [CANDIDATE]: Term 1 election: Requested vote from peers bd843eaf9ad84f73a6ee28a9d1340e72 (127.18.193.3:44489), c4300709c5b7445b8dd9063185d945e0 (127.18.193.2:45317)
I20260807 04:11:50.617188 31457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baac0e0b11224ca1899c09ed6749d72b" candidate_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4300709c5b7445b8dd9063185d945e0"
I20260807 04:11:50.617341 31596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baac0e0b11224ca1899c09ed6749d72b" candidate_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72"
I20260807 04:11:50.617478 31457 raft_consensus.cc:3037] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:11:50.617661 31596 raft_consensus.cc:3037] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:11:50.621210 31457 raft_consensus.cc:2454] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ec2ddbc02e46b0bb95ee190f3d7d1a in term 1.
I20260807 04:11:50.621739 31238 leader_election.cc:302] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1ec2ddbc02e46b0bb95ee190f3d7d1a, c4300709c5b7445b8dd9063185d945e0; no voters:
I20260807 04:11:50.622138 31673 raft_consensus.cc:2781] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:11:50.622665 31596 raft_consensus.cc:2454] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ec2ddbc02e46b0bb95ee190f3d7d1a in term 1.
I20260807 04:11:50.623198 31673 raft_consensus.cc:686] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 1 LEADER]: Becoming Leader. State: Replica: b1ec2ddbc02e46b0bb95ee190f3d7d1a, State: Running, Role: LEADER
I20260807 04:11:50.624306 31673 consensus_queue.cc:237] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } }
I20260807 04:11:50.630275 31130 catalog_manager.cc:5495] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a reported cstate change: term changed from 0 to 1, leader changed from <none> to b1ec2ddbc02e46b0bb95ee190f3d7d1a (127.18.193.1). New cstate: current_term: 1 leader_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1ec2ddbc02e46b0bb95ee190f3d7d1a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 } health_report { overall_health: UNKNOWN } } }
W20260807 04:11:50.643859 31357 tablet.cc:2369] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:11:50.654207 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.647186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.18.193.1:38063)
0807 04:11:50.647438 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.647443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.647460 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:11:50.647572 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.647576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.647657 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.647782 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:11:50.648306 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.649273 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.651078 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.651587 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.651696 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.652451 (+ 755us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:11:50.653420 (+ 969us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:11:50.653565 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.653704 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.653778 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 04:11:50.664264 31648 tablet.cc:2369] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:11:50.665620 31596 raft_consensus.cc:1261] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 1 FOLLOWER]: Refusing update from remote peer b1ec2ddbc02e46b0bb95ee190f3d7d1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:11:50.665654 31457 raft_consensus.cc:1261] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 1 FOLLOWER]: Refusing update from remote peer b1ec2ddbc02e46b0bb95ee190f3d7d1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:11:50.666877 31678 consensus_queue.cc:1035] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd843eaf9ad84f73a6ee28a9d1340e72" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:11:50.667544 31673 consensus_queue.cc:1035] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4300709c5b7445b8dd9063185d945e0" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:11:50.683259 31681 mvcc.cc:205] Tried to move back new op lower bound from 7315766930072248320 to 7315766929922822144. Current Snapshot: MvccSnapshot[applied={T|T < 7315766930072248320}]
I20260807 04:11:50.704068 31130 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:11:50.704389 31130 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:11:50.709242 31130 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ac2f966457049b68788471344dc0725: Sending DeleteTablet for 3 replicas of tablet baac0e0b11224ca1899c09ed6749d72b
I20260807 04:11:50.710873 31436 tablet_service.cc:1514] Processing DeleteTablet for tablet baac0e0b11224ca1899c09ed6749d72b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:11:50 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:47314
I20260807 04:11:50.710872 31575 tablet_service.cc:1514] Processing DeleteTablet for tablet baac0e0b11224ca1899c09ed6749d72b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:11:50 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:47002
I20260807 04:11:50.710937 31287 tablet_service.cc:1514] Processing DeleteTablet for tablet baac0e0b11224ca1899c09ed6749d72b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:11:50 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:41272
I20260807 04:11:50.713212 31693 tablet_replica.cc:330] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: stopping tablet replica
I20260807 04:11:50.713233 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 31214
I20260807 04:11:50.713907 31693 raft_consensus.cc:2229] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:11:50.714746 31693 raft_consensus.cc:2258] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:11:50.717975 31691 tablet_replica.cc:330] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: stopping tablet replica
I20260807 04:11:50.718524 31691 raft_consensus.cc:2229] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:11:50.719513 31691 raft_consensus.cc:2258] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:11:50.720708 31693 ts_tablet_manager.cc:1905] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:11:50.721889 31691 ts_tablet_manager.cc:1905] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:11:50.729447 31693 ts_tablet_manager.cc:1918] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:11:50.729750 31693 log.cc:1196] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/baac0e0b11224ca1899c09ed6749d72b
I20260807 04:11:50.730433 31693 ts_tablet_manager.cc:1939] T baac0e0b11224ca1899c09ed6749d72b P c4300709c5b7445b8dd9063185d945e0: Deleting consensus metadata
I20260807 04:11:50.733132 31114 catalog_manager.cc:4841] TS c4300709c5b7445b8dd9063185d945e0 (127.18.193.2:45317): tablet baac0e0b11224ca1899c09ed6749d72b (table test-table [id=71f655ff0a8b4121bf0ff510272fda7a]) successfully deleted
I20260807 04:11:50.734818 31692 tablet_replica.cc:330] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: stopping tablet replica
I20260807 04:11:50.735443 31692 raft_consensus.cc:2229] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:11:50.736006 31691 ts_tablet_manager.cc:1918] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:11:50.736285 31691 log.cc:1196] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/baac0e0b11224ca1899c09ed6749d72b
I20260807 04:11:50.736459 31692 raft_consensus.cc:2258] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:11:50.737102 31691 ts_tablet_manager.cc:1939] T baac0e0b11224ca1899c09ed6749d72b P b1ec2ddbc02e46b0bb95ee190f3d7d1a: Deleting consensus metadata
I20260807 04:11:50.739017 31692 ts_tablet_manager.cc:1905] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:11:50.739087 31115 catalog_manager.cc:4841] TS b1ec2ddbc02e46b0bb95ee190f3d7d1a (127.18.193.1:38063): tablet baac0e0b11224ca1899c09ed6749d72b (table test-table [id=71f655ff0a8b4121bf0ff510272fda7a]) successfully deleted
Aug 07 04:11:50 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:11:50.746088 31692 ts_tablet_manager.cc:1918] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:11:50.746270 31692 log.cc:1196] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/baac0e0b11224ca1899c09ed6749d72b
I20260807 04:11:50.746894 31692 ts_tablet_manager.cc:1939] T baac0e0b11224ca1899c09ed6749d72b P bd843eaf9ad84f73a6ee28a9d1340e72: Deleting consensus metadata
I20260807 04:11:50.748747 31114 catalog_manager.cc:4841] TS bd843eaf9ad84f73a6ee28a9d1340e72 (127.18.193.3:44489): tablet baac0e0b11224ca1899c09ed6749d72b (table test-table [id=71f655ff0a8b4121bf0ff510272fda7a]) successfully deleted
I20260807 04:11:50.755190 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.715482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41296 (local address 127.18.193.1:38063)
0807 04:11:50.715757 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.715766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.728409 (+ 12643us) server_negotiation.cc:407] Connection header received
0807 04:11:50.729004 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.729010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.729115 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.729198 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:50.731173 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.735017 (+ 3844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.737899 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.739229 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.739405 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.746775 (+ 7370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:50.746791 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:50.746811 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:50.746874 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:50.751809 (+ 4935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.752734 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.752743 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.752747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.752932 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.753332 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.753338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.753341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.753827 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:50.754200 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.754425 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.754718 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 04:11:50.755997 31351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:11:50.857599 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 31214
I20260807 04:11:50.879444 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 31364
Aug 07 04:11:50 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:11:50.897213 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:50.880402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.18.193.2:45317)
0807 04:11:50.880701 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:50.880710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:50.880737 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:11:50.880811 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:50.880818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:50.880896 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:50.881078 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:50.882639 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.883849 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.885466 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:50.886126 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:50.886283 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:50.891298 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:50.891315 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:50.891332 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:50.891370 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:50.894516 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.895217 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.895224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.895227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.895356 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:50.895768 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:50.895771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:50.895773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:50.896129 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:50.896346 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:50.896685 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 04:11:50.896854 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 04:11:50.898082 31500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:11:50.990981 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 31364
I20260807 04:11:51.011062 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 31511
Aug 07 04:11:51 dist-test-slave-8n49 krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:11:51.027486 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:11:51.011877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47006 (local address 127.18.193.3:44489)
0807 04:11:51.012135 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 04:11:51.012141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:11:51.012160 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:11:51.012292 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:11:51.012296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:11:51.012362 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:11:51.012460 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:11:51.013831 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:51.014569 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:51.015918 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:11:51.016309 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:11:51.016404 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:11:51.021194 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:11:51.021209 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:11:51.021226 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:11:51.021261 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:11:51.024239 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:51.025073 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:51.025080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:51.025083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:51.025208 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:11:51.025841 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:11:51.025845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:11:51.025847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:11:51.026264 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:11:51.026600 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 04:11:51.026898 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 04:11:51.027080 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 04:11:51.028131 31642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:11:51.120332 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 31511
I20260807 04:11:51.139406 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 31097
I20260807 04:11:51.140694 31162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:11:51.257110 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 31097
I20260807 04:11:51.282744 19204 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 04:11:51.629568 19204 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 04:11:51.629925 19204 mini_ranger.cc:66] Stopping Ranger...
I20260807 04:11:51.976125 19204 mini_ranger.cc:68] Stopped Ranger
2026-08-07 04:11:51.976 UTC [30655] LOG: received smart shutdown request
2026-08-07 04:11:51.980 UTC [30655] LOG: background worker "logical replication launcher" (PID 30664) exited with exit code 1
2026-08-07 04:11:51.981 UTC [30659] LOG: shutting down
2026-08-07 04:11:52.199 UTC [30655] LOG: database system is shut down
2026-08-07T04:11:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54881 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:11:52 dist-test-slave-8n49 krb5kdc[31707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:11:52 dist-test-slave-8n49 krb5kdc[31707](info): set up 2 sockets
Aug 07 04:11:52 dist-test-slave-8n49 krb5kdc[31707](info): commencing operation
krb5kdc: starting...
W20260807 04:11:53.511837 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:11:53 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:11:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/rangeradmin_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/rangeradmin_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/rangerlookup_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/rangerlookup_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
I20260807 04:11:53.904726 19204 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5Tobyc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/postgres -l logfile start
2026-08-07 04:11:56.628 UTC [31739] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 04:11:56.628 UTC [31739] LOG: listening on IPv4 address "127.18.193.20", port 36635
2026-08-07 04:11:56.632 UTC [31739] LOG: listening on Unix socket "/tmp/.s.PGSQL.36635"
2026-08-07 04:11:56.654 UTC [31742] LOG: database system was shut down at 2026-08-07 04:11:55 UTC
2026-08-07 04:11:56.658 UTC [31739] LOG: database system is ready to accept connections
I20260807 04:11:57.723384 19204 mini_postgres.cc:96] Postgres bound to 36635
2026-08-07 04:11:57.745 UTC [31751] FATAL: database "slave" does not exist
127.18.193.20:36635 - accepting connections
I20260807 04:11:57.746063 19204 mini_ranger.cc:161] Starting Ranger...
I20260807 04:11:57.777374 19204 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 04:11:58.322183 19204 mini_ranger.cc:87] Created ranger Postgres database
I20260807 04:11:58.322288 19204 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-admin
2026-08-07 04:11:58,629 [I] DB FLAVOR :POSTGRES
2026-08-07 04:11:58,630 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:11:58,630 [I] Checking connection..
2026-08-07 04:11:58,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 04:11:58,883 [I] Checking connection passed.
2026-08-07 04:11:58,883 [I] --------- Verifying version history table ---------
2026-08-07 04:11:58,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:11:59,156 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:11:59,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:11:59,424 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:11:59,425 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 04:11:59,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 04:11:59.677 UTC [31819] WARNING: there is no transaction in progress
2026-08-07 04:11:59,684 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 04:11:59,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:11:59,960 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 04:11:59,961 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:11:59,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 04:12:00,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 04:12:00,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8n49.c.gcp-upstream.internal','N') ;"
2026-08-07 04:12:00,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:12:00,987 [I] Table x_portal_user does not exist in database ranger
2026-08-07 04:12:00,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:12:01,236 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 04:12:01,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:12:01,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:12:01,744 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 04:12:01,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 04:12:02.779 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.796 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.814 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.831 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.849 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.867 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.925 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.932 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.949 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.960 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.971 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.985 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:02.995 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:03.002 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:03.010 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:03.017 UTC [31934] WARNING: there is no transaction in progress
2026-08-07 04:12:03,716 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 04:12:03,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 04:12:03,967 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 04:12:03,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:12:04,238 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 04:12:04,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:12:04,500 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 04:12:04,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:12:04,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:12:05,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8n49.c.gcp-upstream.internal';"
2026-08-07 04:12:05,240 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 04:12:05,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 04:12:05,496 [I] DB_PATCHES have already been applied
I20260807 04:12:05.502133 19204 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-admin:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5Tobyc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 04:12:05.502228 19204 mini_ranger.cc:193] Using host: 127.18.193.20
I20260807 04:12:05.523422 19204 mini_ranger.cc:227] Ranger admin URL: http://127.18.193.20:37825
Aug 07, 2026 4:12:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:12:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:12:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 4:12:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 4:12:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 4:12:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 4:12:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 04:12:06.715255 19204 mini_ranger.cc:160] Time spent starting Ranger: real 8.969s user 0.000s sys 0.054s
Aug 07, 2026 4:12:27 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 4:12:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 4:12:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 4:12:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 4:12:28 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 04:12:28.748847 19204 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.193.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.193.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/rangerkms_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.193.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/rangerkms_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.193.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.193.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.193.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/HTTP_127.18.193.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 04:12:28 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075948, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 04:12:28.953770 19204 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 04:12:28.978674 19204 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 04:12:29.953737 19204 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 04:12:29.953866 19204 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms
I20260807 04:12:29.953938 19204 mini_ranger_kms.cc:227] Using postgres at 127.18.193.20:36635
2026-08-07 04:12:30,339 [I] DB FLAVOR :POSTGRES
2026-08-07 04:12:30,340 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:12:30,340 [I] Checking connection
2026-08-07 04:12:30,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 04:12:30,612 [I] connection success
2026-08-07 04:12:30,612 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 04:12:30,613 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 04:12:30,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 04:12:30,859 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 04:12:30,859 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:12:30,860 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 04:12:30,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.193.20:36635/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 04:12:31,134 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 04:12:31.615700 19204 mini_ranger_kms.cc:310] Created kms service
I20260807 04:12:31.851023 19204 mini_ranger_kms.cc:326] Created kudu user
I20260807 04:12:31.901865 19204 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 04:12:31 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075948, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:32.311638 19204 mini_ranger_kms.cc:384] Added ranger policy
I20260807 04:12:32.311995 19204 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms:/tmp/dist-test-task5Tobyc/build/asan/bin/postgresql.jar:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5Tobyc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 04:12:32.312098 19204 mini_ranger_kms.cc:240] Using host: 127.18.193.20
I20260807 04:12:32.338088 19204 mini_ranger_kms.cc:276] Ranger KMS PID: 32131
I20260807 04:12:32.338244 19204 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.193.20:41833
Aug 07, 2026 4:12:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:12:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:12:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 4:12:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 4:12:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 4:12:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 04:12:34.685019 19204 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.731s user 0.071s sys 0.346s
I20260807 04:12:34.685277 19204 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 04:12:34.685451 19204 mini_ranger_kms.cc:401] 127.18.193.20:41833/kms/v1/keys
Aug 07 04:12:34 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786075954, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.193.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 04:12:35 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075954, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.193.20@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
Aug 07 04:12:35 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075954, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.193.20@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
Aug 07 04:12:37 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:12:37.921170 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33175
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:33175
--ranger_config_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:38.260027 32183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:38.260324 32183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:38.260521 32183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:38.267704 32183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:38.267807 32183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:38.267854 32183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:38.267942 32183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:38.268016 32183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:38.278251 32183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.193.20:41833/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:33175
--ranger_config_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:38.279629 32183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:38.280998 32183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:38.292929 32188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:38.292987 32189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:38.293009 32191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:38.293478 32183 server_base.cc:1034] running on GCE node
I20260807 04:12:38.294404 32183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:38.295724 32183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:38.296926 32183 hybrid_clock.cc:648] HybridClock initialized: now 1786075958296874 us; error 46 us; skew 500 ppm
I20260807 04:12:38.297288 32183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:38 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075958, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:38.300410 32183 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:12:38.301992 32183 webserver.cc:460] Webserver started at http://127.18.193.62:36669/ using document root <none> and password file <none>
I20260807 04:12:38.302655 32183 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:38.302800 32183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:38.303112 32183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:12:38 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075958, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:38.354204 32183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "288d718b558f48cea3b1e54d5d0586c5"
format_stamp: "Formatted at 2026-08-07 04:12:38 on dist-test-slave-8n49"
server_key: "711ec0ea103a819409d5a4bd61178824"
server_key_iv: "5d3ab02b0dca901a31feb02c06326c51"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:38.355093 32183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "288d718b558f48cea3b1e54d5d0586c5"
format_stamp: "Formatted at 2026-08-07 04:12:38 on dist-test-slave-8n49"
server_key: "711ec0ea103a819409d5a4bd61178824"
server_key_iv: "5d3ab02b0dca901a31feb02c06326c51"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:38.360980 32183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:12:38.379091 32198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:38.380694 32183 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:12:38.380894 32183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "288d718b558f48cea3b1e54d5d0586c5"
format_stamp: "Formatted at 2026-08-07 04:12:38 on dist-test-slave-8n49"
server_key: "711ec0ea103a819409d5a4bd61178824"
server_key_iv: "5d3ab02b0dca901a31feb02c06326c51"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:38.381140 32183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:38.420641 32183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:38.425017 32183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:38.425339 32183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:38.443872 32183 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:33175
I20260807 04:12:38.443912 32251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:33175 every 8 connection(s)
I20260807 04:12:38.445242 32183 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:12:38.448868 32252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:38.451948 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 32183
I20260807 04:12:38.452188 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:12:38.453102 19204 external_mini_cluster.cc:1428] Setting key 5b34eac03a10abbe23ff8e974b3da20e
I20260807 04:12:38.462541 32252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Bootstrap starting.
I20260807 04:12:38.466796 32252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:38.468278 32252 log.cc:824] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:12:38 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:38.471777 32252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: No bootstrap required, opened a new log
I20260807 04:12:38.477896 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:38.455549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.18.193.62:33175)
0807 04:12:38.456623 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:38.456635 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:38.456679 (+ 44us) server_negotiation.cc:407] Connection header received
0807 04:12:38.458318 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:38.458340 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:38.458716 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:38.459050 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:38.460796 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:38.462251 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:38.464128 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:38.464955 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:38.465109 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:38.470290 (+ 5181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:38.470299 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:38.470315 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:38.470357 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:38.473550 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:38.474322 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:38.474330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:38.474337 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:38.474513 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:38.474919 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:38.474924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:38.474930 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:38.475537 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:38.475855 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:38.476359 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 04:12:38.476820 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":557,"threads_started":1}
I20260807 04:12:38.480512 32252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:38.481112 32252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:38.481226 32252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 288d718b558f48cea3b1e54d5d0586c5, State: Initialized, Role: FOLLOWER
I20260807 04:12:38.481827 32252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:38.482033 32252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:12:38.482134 32252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:12:38.482261 32252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:38.484584 32252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:38.485167 32252 leader_election.cc:302] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 288d718b558f48cea3b1e54d5d0586c5; no voters:
I20260807 04:12:38.485953 32252 leader_election.cc:288] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:12:38.486065 32257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:12:38.487525 32257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 LEADER]: Becoming Leader. State: Replica: 288d718b558f48cea3b1e54d5d0586c5, State: Running, Role: LEADER
I20260807 04:12:38.488149 32257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:38.488472 32252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:12:38.491032 32259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 288d718b558f48cea3b1e54d5d0586c5. Latest consensus state: current_term: 1 leader_uuid: "288d718b558f48cea3b1e54d5d0586c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } } }
I20260807 04:12:38.491367 32259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:38.491580 32258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "288d718b558f48cea3b1e54d5d0586c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } } }
I20260807 04:12:38.491950 32258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:38.496290 32268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:12:38.496493 32252 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 04:12:38.509207 32268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:12:38.520999 32268 catalog_manager.cc:1347] Generated new cluster ID: d1ad083853ee4ccc87b76e49a99dbc02
I20260807 04:12:38.521145 32268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:12:38.568810 32268 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:12:38.570822 32268 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:12:38.579944 32268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Generated new TSK 0
I20260807 04:12:38.581113 32268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:12:39.327971 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33175
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:12:39 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786075959, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 04:12:39.789561 32300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:39.790002 32300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:39.790169 32300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:39.802003 32300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:39.802225 32300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:39.802467 32300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:12:39.818512 32300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.193.20:41833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:39.820428 32300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:39.822237 32300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:39.840201 32310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:39.840814 32311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:39.845573 32314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:39.847141 32300 server_base.cc:1034] running on GCE node
I20260807 04:12:39.848002 32300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:39.849390 32300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:39.850705 32300 hybrid_clock.cc:648] HybridClock initialized: now 1786075959850633 us; error 65 us; skew 500 ppm
I20260807 04:12:39.851104 32300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:39 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075959, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:39.855553 32300 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:12:39.857676 32300 webserver.cc:460] Webserver started at http://127.18.193.1:46121/ using document root <none> and password file <none>
I20260807 04:12:39.858755 32300 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:39.858912 32300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:39.859314 32300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:12:39 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075959, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:39.882299 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "3443a22037da4e39b7baa1b70cd8b1d0"
format_stamp: "Formatted at 2026-08-07 04:12:39 on dist-test-slave-8n49"
server_key: "ae95b3310f113de7884a73b3ae7a4063"
server_key_iv: "f5af1704f3627e300f11476a35e0f78a"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:39.883659 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "3443a22037da4e39b7baa1b70cd8b1d0"
format_stamp: "Formatted at 2026-08-07 04:12:39 on dist-test-slave-8n49"
server_key: "ae95b3310f113de7884a73b3ae7a4063"
server_key_iv: "f5af1704f3627e300f11476a35e0f78a"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:39.893087 32300 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260807 04:12:39.927990 32322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:39.930383 32300 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.008s sys 0.002s
I20260807 04:12:39.930783 32300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "3443a22037da4e39b7baa1b70cd8b1d0"
format_stamp: "Formatted at 2026-08-07 04:12:39 on dist-test-slave-8n49"
server_key: "ae95b3310f113de7884a73b3ae7a4063"
server_key_iv: "f5af1704f3627e300f11476a35e0f78a"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:39.931329 32300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:39.964318 32300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:39.970145 32300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:39.970544 32300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:39.983457 32300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:39.985677 32300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:39.985790 32300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:39.985991 32300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:39.986066 32300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:40.069651 32300 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:38007
I20260807 04:12:40.069700 32439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:38007 every 8 connection(s)
I20260807 04:12:40.071481 32300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:12:40.075990 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 32300
I20260807 04:12:40.076265 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:12:40.077298 19204 external_mini_cluster.cc:1428] Setting key 84bf991b253b17cda260599984506a49
Aug 07 04:12:40 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075959, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
I20260807 04:12:40.148241 32443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:40.076900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:41579 (local address 127.18.193.62:33175)
0807 04:12:40.113587 (+ 36687us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:40.113595 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:40.116669 (+ 3074us) server_negotiation.cc:407] Connection header received
0807 04:12:40.122975 (+ 6306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:40.122983 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:40.123135 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:40.123249 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:40.126393 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.127592 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.131093 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.131783 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.131894 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:40.136622 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:40.136630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:40.136635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:40.136679 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:40.140477 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:40.141811 (+ 1334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:40.141820 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:40.141824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:40.142016 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:40.145672 (+ 3656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:40.145681 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:40.145687 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:40.146120 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:40.146273 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:40.147402 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0807 04:12:40.147690 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36522,"thread_start_us":36698,"threads_started":1}
I20260807 04:12:40.151113 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:40.075305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33175 (local address 127.18.193.1:41579)
0807 04:12:40.114418 (+ 39113us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:40.114483 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:40.122573 (+ 8090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:40.123849 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:40.123862 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:40.125228 (+ 1366us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:40.126149 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:40.126185 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.128701 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.128708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:40.130831 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:40.130849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.132266 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.132274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:40.132647 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:40.133490 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:40.133525 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:40.136282 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:40.140914 (+ 4632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:40.140925 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:40.140944 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:40.141569 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:40.142190 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:40.142195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:40.142198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:40.145497 (+ 3299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:40.146382 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:40.146393 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:40.146913 (+ 520us) client_negotiation.cc:769] Sending connection context
0807 04:12:40.148871 (+ 1958us) client_negotiation.cc:240] Negotiation successful
0807 04:12:40.149532 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38720,"thread_start_us":45142,"threads_started":1}
I20260807 04:12:40.153412 32440 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33175
I20260807 04:12:40.153805 32440 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:40.154757 32440 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
I20260807 04:12:40.157497 32217 ts_manager.cc:194] Registered new tserver with Master: 3443a22037da4e39b7baa1b70cd8b1d0 (127.18.193.1:38007)
I20260807 04:12:40.159643 32217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:41579
Aug 07 04:12:40 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075959, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
W20260807 04:12:40.193744 19204 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.193.2: real 0.112s user 0.001s sys 0.021s
I20260807 04:12:40.194779 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33175
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:12:40 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075959, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
W20260807 04:12:40.690387 32446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:40.690879 32446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:40.691058 32446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:40.702509 32446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:40.702767 32446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:40.703006 32446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:12:40.718570 32446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.193.20:41833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:40.720679 32446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:40.722450 32446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:40.762115 32453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:40.769611 32454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:40.770978 32457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:40.771497 32446 server_base.cc:1034] running on GCE node
I20260807 04:12:40.772693 32446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:40.784922 32446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:40.786326 32446 hybrid_clock.cc:648] HybridClock initialized: now 1786075960786212 us; error 116 us; skew 500 ppm
I20260807 04:12:40.786765 32446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:40 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075960, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:40.791280 32446 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:12:40.793668 32446 webserver.cc:460] Webserver started at http://127.18.193.2:42791/ using document root <none> and password file <none>
I20260807 04:12:40.794683 32446 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:40.794831 32446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:40.795187 32446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:12:40 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075960, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:40.812928 32446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "fa0f490ee8b0431fa1614d9ac1f51553"
format_stamp: "Formatted at 2026-08-07 04:12:40 on dist-test-slave-8n49"
server_key: "b52a36c643171e349cbf9f7ce3f41d7a"
server_key_iv: "5b32dde884f8d7bdec52addd6b097475"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:40.814184 32446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "fa0f490ee8b0431fa1614d9ac1f51553"
format_stamp: "Formatted at 2026-08-07 04:12:40 on dist-test-slave-8n49"
server_key: "b52a36c643171e349cbf9f7ce3f41d7a"
server_key_iv: "5b32dde884f8d7bdec52addd6b097475"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:40.833508 32446 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.013s sys 0.004s
I20260807 04:12:40.861166 32471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:40.863166 32446 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.004s
I20260807 04:12:40.863381 32446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "fa0f490ee8b0431fa1614d9ac1f51553"
format_stamp: "Formatted at 2026-08-07 04:12:40 on dist-test-slave-8n49"
server_key: "b52a36c643171e349cbf9f7ce3f41d7a"
server_key_iv: "5b32dde884f8d7bdec52addd6b097475"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:40.863677 32446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:40.904330 32446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:40.908644 32446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:40.908973 32446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:40.910090 32446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:40.911631 32446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:40.911707 32446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:40.911850 32446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:40.911908 32446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:40.952159 32446 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:40671
I20260807 04:12:40.952180 32588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:40671 every 8 connection(s)
I20260807 04:12:40.953590 32446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:12:40.960098 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 32446
I20260807 04:12:40.960290 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:12:40.960908 19204 external_mini_cluster.cc:1428] Setting key 9f001cec693d341eb695b556c9de3750
Aug 07 04:12:40 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075960, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:40.978621 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:40.956626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:37429 (local address 127.18.193.62:33175)
0807 04:12:40.957092 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:40.957097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:40.958327 (+ 1230us) server_negotiation.cc:407] Connection header received
0807 04:12:40.960210 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:40.960216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:40.960322 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:40.960447 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:40.962259 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.963690 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.966492 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.967195 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.967360 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:40.971783 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:40.971789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:40.971792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:40.971827 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:40.975218 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:40.976001 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:40.976006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:40.976010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:40.976147 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:40.976674 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:40.976680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:40.976683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:40.976978 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:40.977144 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:40.977975 (+ 831us) server_negotiation.cc:299] Negotiation successful
0807 04:12:40.978221 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":244,"threads_started":1}
I20260807 04:12:40.979647 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:40.957336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33175 (local address 127.18.193.2:37429)
0807 04:12:40.958137 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:40.958183 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:40.959913 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:40.960749 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:40.960760 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:40.961301 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:40.962059 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:40.962086 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.963902 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.963910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:40.966288 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:40.966308 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:40.967473 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:40.967483 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:40.967845 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:40.968804 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:40.968842 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:40.971510 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:40.975364 (+ 3854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:40.975375 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:40.975394 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:40.975861 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:40.976276 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:40.976281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:40.976285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:40.976511 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:40.977158 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:40.977169 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:40.977525 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 04:12:40.977865 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 04:12:40.978432 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":332,"threads_started":1}
I20260807 04:12:40.981314 32589 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33175
I20260807 04:12:40.981644 32589 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:40.982381 32589 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
I20260807 04:12:40.983994 32217 ts_manager.cc:194] Registered new tserver with Master: fa0f490ee8b0431fa1614d9ac1f51553 (127.18.193.2:40671)
I20260807 04:12:40.985359 32217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:37429
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:12:41.025581 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33175
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:12:41.163751 32440 heartbeater.cc:503] Master 127.18.193.62:33175 was elected leader, sending a full tablet report...
W20260807 04:12:41.362530 32596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:41.362841 32596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:41.363020 32596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:41.372570 32596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:41.372745 32596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:41.372992 32596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:12:41.384034 32596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.193.20:41833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:41.385515 32596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:41.386780 32596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:41.399989 32603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:41.400096 32602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:41.400125 32605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:41.400964 32596 server_base.cc:1034] running on GCE node
I20260807 04:12:41.401559 32596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:41.402515 32596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:41.403753 32596 hybrid_clock.cc:648] HybridClock initialized: now 1786075961403697 us; error 42 us; skew 500 ppm
I20260807 04:12:41.404093 32596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:41 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075961, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:41.407266 32596 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:12:41.408829 32596 webserver.cc:460] Webserver started at http://127.18.193.3:46553/ using document root <none> and password file <none>
I20260807 04:12:41.409500 32596 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:41.409662 32596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:41.409982 32596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:12:41 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075961, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:41.426046 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "70496a1b310f43ce91f83669accc2073"
format_stamp: "Formatted at 2026-08-07 04:12:41 on dist-test-slave-8n49"
server_key: "019c70a1de9424ebe065d335d8a5b21d"
server_key_iv: "00e900cdda470ab4e3556f498941ee22"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:41.426977 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "70496a1b310f43ce91f83669accc2073"
format_stamp: "Formatted at 2026-08-07 04:12:41 on dist-test-slave-8n49"
server_key: "019c70a1de9424ebe065d335d8a5b21d"
server_key_iv: "00e900cdda470ab4e3556f498941ee22"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:41.432997 32596 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 04:12:41.451193 32612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:41.452955 32596 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:12:41.453127 32596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "70496a1b310f43ce91f83669accc2073"
format_stamp: "Formatted at 2026-08-07 04:12:41 on dist-test-slave-8n49"
server_key: "019c70a1de9424ebe065d335d8a5b21d"
server_key_iv: "00e900cdda470ab4e3556f498941ee22"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:41.453404 32596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:41.473219 32596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:41.477362 32596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:41.477676 32596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:41.478734 32596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:41.480350 32596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:41.480427 32596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:41.480607 32596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:41.480670 32596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 04:12:41.516392 32596 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33297
I20260807 04:12:41.516494 32731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33297 every 8 connection(s)
I20260807 04:12:41.517823 32596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:12:41.524209 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 32596
I20260807 04:12:41.524441 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:12:41.524986 19204 external_mini_cluster.cc:1428] Setting key 2bb65a8bf4be0ec1ca4ff91ff28f9837
Aug 07 04:12:41 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075961, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:41.540580 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:41.520585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:48899 (local address 127.18.193.62:33175)
0807 04:12:41.521099 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:41.521104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:41.522138 (+ 1034us) server_negotiation.cc:407] Connection header received
0807 04:12:41.524814 (+ 2676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:41.524820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:41.525028 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:41.525192 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:41.527117 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.527904 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.529863 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.530311 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.530426 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:41.534044 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:41.534049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:41.534053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:41.534087 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:41.536776 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.537738 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.537744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.537752 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.537916 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.538378 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.538382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.538385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.538734 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:41.538912 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:41.539933 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0807 04:12:41.540182 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":228,"threads_started":1}
I20260807 04:12:41.541765 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:41.521181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33175 (local address 127.18.193.3:48899)
0807 04:12:41.521950 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:41.521995 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:41.524430 (+ 2435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:41.525546 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:41.525559 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:41.526027 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:41.526862 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:41.526889 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.528083 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.528089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:41.529705 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:41.529718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.530558 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.530564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:41.530783 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:41.531443 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:41.531468 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:41.533774 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:41.536947 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:41.536954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:41.536969 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:41.537459 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:41.538027 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:41.538031 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:41.538033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:41.538227 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:41.538949 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:41.538957 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:41.539383 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 04:12:41.539793 (+ 410us) client_negotiation.cc:240] Negotiation successful
0807 04:12:41.540345 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":319,"threads_started":1}
I20260807 04:12:41.543210 32732 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33175
I20260807 04:12:41.543550 32732 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:41.544430 32732 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
I20260807 04:12:41.545980 32217 ts_manager.cc:194] Registered new tserver with Master: 70496a1b310f43ce91f83669accc2073 (127.18.193.3:33297)
I20260807 04:12:41.546938 32217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:48899
I20260807 04:12:41.553853 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:12:41.555627 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 32300
Aug 07 04:12:41 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:12:41.573199 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:41.556870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.18.193.1:38007)
0807 04:12:41.557567 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:41.557573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:41.557599 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:12:41.557656 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:41.557663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:41.557867 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:41.558057 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:41.559537 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.560679 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.562050 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.562629 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.562731 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:41.567100 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:41.567111 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:41.567126 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:41.567166 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:41.570285 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.570974 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.570981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.570983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.571145 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.571587 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.571591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.571593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.571951 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:41.572195 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:41.572483 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 04:12:41.572775 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":379,"threads_started":1}
I20260807 04:12:41.574533 32435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:41.661279 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 32300
I20260807 04:12:41.678855 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 32446
Aug 07 04:12:41 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:12:41.704814 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:41.679854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.18.193.2:40671)
0807 04:12:41.680516 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:41.680523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:41.680590 (+ 67us) server_negotiation.cc:407] Connection header received
0807 04:12:41.680695 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:41.680702 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:41.680931 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:41.681071 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:41.682931 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.684446 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.686909 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.687496 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.687612 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:41.697740 (+ 10128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:41.697760 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:41.697778 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:41.697836 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:41.701391 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.702244 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.702253 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.702255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.702410 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.702815 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.702820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.702823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.703233 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:41.703577 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:41.703905 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 04:12:41.704241 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":331,"threads_started":1}
I20260807 04:12:41.706725 32584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:41.793920 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 32446
I20260807 04:12:41.811512 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 32596
Aug 07 04:12:41 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:12:41.828058 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:41.812472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.18.193.3:33297)
0807 04:12:41.813118 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:41.813125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:41.813154 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:12:41.813213 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:41.813220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:41.813432 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:41.813613 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:41.814882 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.815926 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.817278 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:41.817846 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:41.817989 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:41.822657 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:41.822670 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:41.822685 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:41.822725 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:41.825489 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.826163 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.826170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.826172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.826298 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:41.826665 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:41.826668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:41.826670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:41.827019 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:41.827261 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:41.827565 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 04:12:41.827765 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":342,"threads_started":1}
I20260807 04:12:41.829615 32727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:41.914494 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 32596
I20260807 04:12:41.931958 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 32183
I20260807 04:12:41.933743 32247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:42.044684 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 32183
I20260807 04:12:42.070257 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33175
--webserver_interface=127.18.193.62
--webserver_port=36669
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:33175
--ranger_config_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:42.406597 32746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:42.406898 32746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:42.407086 32746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:42.414482 32746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:42.414639 32746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:42.414746 32746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:42.414840 32746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:42.414927 32746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:42.425442 32746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:33175
--ranger_config_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:33175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=36669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:42.426815 32746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:42.428174 32746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:42.440526 32754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:42.440605 32752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:42.440747 32751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:42.441309 32746 server_base.cc:1034] running on GCE node
I20260807 04:12:42.442215 32746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:42.443711 32746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:42.444922 32746 hybrid_clock.cc:648] HybridClock initialized: now 1786075962444873 us; error 40 us; skew 500 ppm
I20260807 04:12:42.445330 32746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:42 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075962, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:42.453099 32746 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:12:42.454916 32746 webserver.cc:460] Webserver started at http://127.18.193.62:36669/ using document root <none> and password file <none>
I20260807 04:12:42.455739 32746 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:42.455889 32746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:12:42 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075962, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:42.506902 32746 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:12:42.510177 32762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:42.511752 32746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:12:42.511961 32746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "288d718b558f48cea3b1e54d5d0586c5"
format_stamp: "Formatted at 2026-08-07 04:12:38 on dist-test-slave-8n49"
server_key: "711ec0ea103a819409d5a4bd61178824"
server_key_iv: "5d3ab02b0dca901a31feb02c06326c51"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:42.513075 32746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:42.545379 32746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:42.549292 32746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:42.549604 32746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:42.568923 32746 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:33175
I20260807 04:12:42.568948 349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:33175 every 8 connection(s)
I20260807 04:12:42.570541 32746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:12:42.577507 350 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:12:42.578474 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 32746
I20260807 04:12:42.579566 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:38007
--local_ip_for_outbound_sockets=127.18.193.1
--tserver_master_addrs=127.18.193.62:33175
--webserver_port=46121
--webserver_interface=127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:12:42.582684 350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Bootstrap starting.
I20260807 04:12:42.603889 350 log.cc:824] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:42.613127 350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:12:42.613705 350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Bootstrap complete.
I20260807 04:12:42.623335 350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:42.624188 350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 288d718b558f48cea3b1e54d5d0586c5, State: Initialized, Role: FOLLOWER
I20260807 04:12:42.624912 350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:42.625159 350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:12:42.625276 350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:12:42.625478 350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:12:42.629407 350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:42.629952 350 leader_election.cc:302] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 288d718b558f48cea3b1e54d5d0586c5; no voters:
I20260807 04:12:42.630647 350 leader_election.cc:288] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:12:42.630795 354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:12:42.632195 354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [term 2 LEADER]: Becoming Leader. State: Replica: 288d718b558f48cea3b1e54d5d0586c5, State: Running, Role: LEADER
I20260807 04:12:42.632845 354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } }
I20260807 04:12:42.633039 350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:12:42.636524 356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 288d718b558f48cea3b1e54d5d0586c5. Latest consensus state: current_term: 2 leader_uuid: "288d718b558f48cea3b1e54d5d0586c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } } }
I20260807 04:12:42.637070 356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:42.637708 355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "288d718b558f48cea3b1e54d5d0586c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288d718b558f48cea3b1e54d5d0586c5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 33175 } } }
I20260807 04:12:42.638072 355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:42.645972 362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:12:42.652171 362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:12:42.653048 350 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 04:12:42.653828 362 catalog_manager.cc:1259] Loaded cluster ID: d1ad083853ee4ccc87b76e49a99dbc02
I20260807 04:12:42.654001 362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:12:42.658020 362 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:12:42.659521 362 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Loaded TSK: 0
I20260807 04:12:42.661139 362 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:12:43.044767 352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:43.045588 352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:43.045847 352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:43.059221 352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:43.059512 352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:43.059787 352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:12:43.077663 352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:38007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=46121
--tserver_master_addrs=127.18.193.62:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:43.079799 352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:43.081758 352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:43.101940 387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:43.102532 388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:43.114113 390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:43.114688 352 server_base.cc:1034] running on GCE node
I20260807 04:12:43.115749 352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:43.119014 352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:43.126550 352 hybrid_clock.cc:648] HybridClock initialized: now 1786075963126462 us; error 74 us; skew 500 ppm
I20260807 04:12:43.127010 352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:43 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:43.131682 352 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:12:43.139040 352 webserver.cc:460] Webserver started at http://127.18.193.1:46121/ using document root <none> and password file <none>
I20260807 04:12:43.140198 352 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:43.140396 352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:12:43 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:43.172299 352 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:12:43.181562 400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:43.183724 352 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.004s
I20260807 04:12:43.183985 352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "3443a22037da4e39b7baa1b70cd8b1d0"
format_stamp: "Formatted at 2026-08-07 04:12:39 on dist-test-slave-8n49"
server_key: "ae95b3310f113de7884a73b3ae7a4063"
server_key_iv: "f5af1704f3627e300f11476a35e0f78a"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:43.185770 352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:43.221748 352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:43.226969 352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:43.227291 352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:43.229602 352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:43.231208 352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:43.231328 352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:43.231459 352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:43.231559 352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:43.301394 352 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:38007
I20260807 04:12:43.301429 518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:38007 every 8 connection(s)
I20260807 04:12:43.303264 352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:12:43.313395 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 352
I20260807 04:12:43.314471 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:40671
--local_ip_for_outbound_sockets=127.18.193.2
--tserver_master_addrs=127.18.193.62:33175
--webserver_port=42791
--webserver_interface=127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:12:43 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:43.354321 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:43.308132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33175 (local address 127.18.193.1:37683)
0807 04:12:43.317201 (+ 9069us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:43.317255 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:43.319216 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:43.325350 (+ 6134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:43.325363 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:43.326063 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:43.326996 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:43.327030 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.330510 (+ 3480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.330518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:43.332526 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:43.332590 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.334337 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.334344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:43.334662 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:43.335375 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:43.335417 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:43.338054 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:43.343576 (+ 5522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:43.343586 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:43.343604 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:43.344226 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:43.345603 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:43.345608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:43.345612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:43.345867 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:43.346987 (+ 1120us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:43.346996 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:43.347460 (+ 464us) client_negotiation.cc:769] Sending connection context
0807 04:12:43.347777 (+ 317us) client_negotiation.cc:240] Negotiation successful
0807 04:12:43.348361 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8735,"thread_start_us":8481,"threads_started":1}
I20260807 04:12:43.359236 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:43.309297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:37683 (local address 127.18.193.62:33175)
0807 04:12:43.323366 (+ 14069us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:43.323389 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:43.323467 (+ 78us) server_negotiation.cc:407] Connection header received
0807 04:12:43.323992 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:43.324016 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:43.324624 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:43.327317 (+ 2693us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:43.327803 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.330262 (+ 2459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.333018 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.334074 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.335128 (+ 1054us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:43.339380 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:43.339390 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:43.339419 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:43.339468 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:43.343377 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:43.344754 (+ 1377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:43.344763 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:43.344772 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:43.345001 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:43.346034 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:43.346038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:43.346042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:43.346816 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:43.356122 (+ 9306us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:43.356495 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 04:12:43.357029 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12283,"thread_start_us":12080,"threads_started":1}
I20260807 04:12:43.363298 519 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33175
I20260807 04:12:43.363648 519 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:43.364758 519 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
W20260807 04:12:43.368983 519 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:33175 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 04:12:43.369277 519 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:43.369645 519 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
I20260807 04:12:43.371292 519 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:43.371623 519 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
W20260807 04:12:43.373085 519 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:12:43.734124 523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:43.734447 523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:43.734615 523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:43.742241 523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:43.742403 523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:43.742609 523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:12:43.753197 523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:40671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=42791
--tserver_master_addrs=127.18.193.62:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:43.754616 523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:43.755903 523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:43.770629 535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:43.770742 538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:43.770519 536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:43.771795 523 server_base.cc:1034] running on GCE node
I20260807 04:12:43.772712 523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:43.774036 523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:43.775355 523 hybrid_clock.cc:648] HybridClock initialized: now 1786075963775257 us; error 95 us; skew 500 ppm
I20260807 04:12:43.775765 523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:43 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:43.780006 523 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:12:43.781704 523 webserver.cc:460] Webserver started at http://127.18.193.2:42791/ using document root <none> and password file <none>
I20260807 04:12:43.782449 523 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:43.782618 523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:12:43 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:43.806190 523 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:12:43.812042 549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:43.814016 523 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260807 04:12:43.814215 523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "fa0f490ee8b0431fa1614d9ac1f51553"
format_stamp: "Formatted at 2026-08-07 04:12:40 on dist-test-slave-8n49"
server_key: "b52a36c643171e349cbf9f7ce3f41d7a"
server_key_iv: "5b32dde884f8d7bdec52addd6b097475"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:43.815724 523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:43.835201 523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:43.840617 523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:43.840975 523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:43.842391 523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:43.844370 523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:43.844498 523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:43.844699 523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:43.844797 523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:43.926203 523 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:40671
I20260807 04:12:43.928287 523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:12:43.926230 665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:40671 every 8 connection(s)
I20260807 04:12:43.934551 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 523
I20260807 04:12:43.935360 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.193.20:41833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:33297
--local_ip_for_outbound_sockets=127.18.193.3
--tserver_master_addrs=127.18.193.62:33175
--webserver_port=46553
--webserver_interface=127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:12:43 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:43.983494 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:43.938276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:45293 (local address 127.18.193.62:33175)
0807 04:12:43.938801 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:43.938808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:43.958218 (+ 19410us) server_negotiation.cc:407] Connection header received
0807 04:12:43.962327 (+ 4109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:43.962337 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:43.962449 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:43.962549 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:43.965049 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.966296 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.969139 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.969954 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.970090 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:43.975228 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:43.975239 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:43.975247 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:43.975301 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:43.979419 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:43.980447 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:43.980454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:43.980458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:43.980634 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:43.981294 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:43.981299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:43.981303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:43.981666 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:43.981805 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:43.982782 (+ 977us) server_negotiation.cc:299] Negotiation successful
0807 04:12:43.983014 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":496,"threads_started":1}
I20260807 04:12:43.986012 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:43.933711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33175 (local address 127.18.193.2:45293)
0807 04:12:43.957962 (+ 24251us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:43.958042 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:43.961931 (+ 3889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:43.963221 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:43.963239 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:43.963926 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:43.964803 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:43.964833 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.966522 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.966531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:43.968912 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:43.968931 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:43.970532 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:43.970551 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:43.970911 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:43.971812 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:43.971856 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:43.974880 (+ 3024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:43.979635 (+ 4755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:43.979649 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:43.979672 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:43.980298 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:43.980888 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:43.980893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:43.980912 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:43.981164 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:43.981895 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:43.981904 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:43.982358 (+ 454us) client_negotiation.cc:769] Sending connection context
0807 04:12:43.984034 (+ 1676us) client_negotiation.cc:240] Negotiation successful
0807 04:12:43.984664 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23804,"thread_start_us":22007,"threads_started":1}
I20260807 04:12:43.988034 666 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33175
I20260807 04:12:43.988350 666 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:43.989248 666 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
I20260807 04:12:43.991307 313 ts_manager.cc:194] Registered new tserver with Master: fa0f490ee8b0431fa1614d9ac1f51553 (127.18.193.2:40671)
I20260807 04:12:43.993300 313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:45293
Aug 07 04:12:44 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786075964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 04:12:44.364144 671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:44.364704 671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:44.364877 671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 04:12:44.373231 519 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:44.373739 519 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
I20260807 04:12:44.375133 313 ts_manager.cc:194] Registered new tserver with Master: 3443a22037da4e39b7baa1b70cd8b1d0 (127.18.193.1:38007)
I20260807 04:12:44.376288 313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:37683
W20260807 04:12:44.377954 671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:44.378136 671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:44.378338 671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:12:44.394197 671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.193.20:41833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:33297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=46553
--tserver_master_addrs=127.18.193.62:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:44.396086 671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:44.397894 671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:44.415292 677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:44.416342 678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:44.426745 671 server_base.cc:1034] running on GCE node
W20260807 04:12:44.428306 680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:44.429525 671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:44.430939 671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:44.432261 671 hybrid_clock.cc:648] HybridClock initialized: now 1786075964432203 us; error 45 us; skew 500 ppm
I20260807 04:12:44.432731 671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:44 dist-test-slave-8n49 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:44.438050 671 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:12:44.444456 671 webserver.cc:460] Webserver started at http://127.18.193.3:46553/ using document root <none> and password file <none>
I20260807 04:12:44.445587 671 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:44.445899 671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:12:44 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
Aug 07 04:12:44 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:44.517652 671 fs_manager.cc:709] Time spent opening directory manager: real 0.032s user 0.012s sys 0.016s
I20260807 04:12:44.527661 688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:44.529785 671 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.009s sys 0.000s
I20260807 04:12:44.530074 671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "70496a1b310f43ce91f83669accc2073"
format_stamp: "Formatted at 2026-08-07 04:12:41 on dist-test-slave-8n49"
server_key: "019c70a1de9424ebe065d335d8a5b21d"
server_key_iv: "00e900cdda470ab4e3556f498941ee22"
server_key_version: "kuduclusterkey@0"
I20260807 04:12:44.531659 671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:44.591013 671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:44.596536 671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:44.596865 671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:44.598917 671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:44.600718 671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:44.600826 671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:44.600930 671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:44.600998 671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:44.670054 671 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33297
I20260807 04:12:44.670109 804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33297 every 8 connection(s)
I20260807 04:12:44.671566 671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:12:44.677747 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 671
Aug 07 04:12:44 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:44.705948 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.682094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.18.193.62:33175)
0807 04:12:44.685461 (+ 3367us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.685467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.685482 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:12:44.685539 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.685543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.685618 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.687825 (+ 2207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:44.688044 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.689211 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.691244 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.692762 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.692876 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.697206 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:44.697212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:44.697215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:44.697267 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:44.701661 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:44.702576 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:44.702582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:44.702585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:44.702787 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:44.703255 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:44.703263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:44.703269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:44.703738 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:44.705188 (+ 1450us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.705320 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.705551 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":3243,"spinlock_wait_cycles":12160,"thread_start_us":3111,"threads_started":1}
I20260807 04:12:44.706818 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.675564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:33175 (local address 127.18.193.3:57437)
0807 04:12:44.677991 (+ 2427us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:44.678045 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:44.684026 (+ 5981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:44.685663 (+ 1637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:44.685677 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:44.686449 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:44.687431 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.687462 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.689192 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.689197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.691284 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.691312 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.692354 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.692360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.692718 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.693589 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:44.693629 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:44.696769 (+ 3140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:44.701803 (+ 5034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:44.701812 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:44.701831 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:44.702460 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:44.702928 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:44.702933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:44.702937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:44.703214 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:44.703974 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:44.703982 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:44.704439 (+ 457us) client_negotiation.cc:769] Sending connection context
0807 04:12:44.704798 (+ 359us) client_negotiation.cc:240] Negotiation successful
0807 04:12:44.705388 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2088,"thread_start_us":5132,"threads_started":1}
I20260807 04:12:44.708622 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.676333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:57437 (local address 127.18.193.62:33175)
0807 04:12:44.682102 (+ 5769us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.682108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.682129 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:12:44.684603 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.684612 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.684729 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.684858 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:44.687672 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.688987 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.691458 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.692100 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.692243 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.697069 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:44.697078 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:44.697082 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:44.697141 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:44.701609 (+ 4468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:44.702581 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:44.702585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:44.702588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:44.702787 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:44.703347 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:44.703354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:44.703360 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:44.703738 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:44.703882 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.707926 (+ 4044us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.708171 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":5650,"spinlock_wait_cycles":6272,"thread_start_us":5531,"threads_started":1}
I20260807 04:12:44.709829 805 heartbeater.cc:347] Connected to a master server at 127.18.193.62:33175
I20260807 04:12:44.710165 805 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:44.710826 805 heartbeater.cc:511] Master 127.18.193.62:33175 requested a full tablet report, sending...
I20260807 04:12:44.712296 312 ts_manager.cc:194] Registered new tserver with Master: 70496a1b310f43ce91f83669accc2073 (127.18.193.3:33297)
I20260807 04:12:44.713336 312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:57437
I20260807 04:12:44.724006 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:12:44.742955 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.727515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.18.193.62:33175)
0807 04:12:44.727789 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.727798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.728344 (+ 546us) server_negotiation.cc:407] Connection header received
0807 04:12:44.728576 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.728582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.728664 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.728771 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:44.730321 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.731072 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.732455 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.733059 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.733192 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.736093 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:44.736099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:44.736102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:44.736134 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:44.739430 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:44.740340 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:44.740346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:44.740349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:44.740508 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:44.741032 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:44.741037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:44.741041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:44.741455 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:44.741706 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.742354 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.742609 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 04:12:44.749804 312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:12:44.757546 312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:12:44.817147 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.774298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:40671 (local address 127.0.0.1:57816)
0807 04:12:44.786503 (+ 12205us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:44.786550 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:44.786750 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:44.806804 (+ 20054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:44.806813 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:44.806856 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:44.807284 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.807299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.809574 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.809580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.812111 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.812130 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.815942 (+ 3812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.815949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.816187 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.816257 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 04:12:44.816402 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 04:12:44.816668 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11946,"threads_started":1}
I20260807 04:12:44.826246 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.774729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.18.193.2:40671)
0807 04:12:44.805880 (+ 31151us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.805896 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.805945 (+ 49us) server_negotiation.cc:407] Connection header received
0807 04:12:44.806093 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.806106 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.806486 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.806785 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:44.807501 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.809120 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.813742 (+ 4622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.815659 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.824699 (+ 9040us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.824970 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.825075 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.825269 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30880,"thread_start_us":32248,"threads_started":1}
I20260807 04:12:44.831032 599 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb968841889421bb33f65bb628a1b70 (DEFAULT_TABLE table=test-table [id=fd14bcb9c9aa4dbf8ae0f577d4a95662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:12:44.832638 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.779502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38007 (local address 127.0.0.1:39824)
0807 04:12:44.817457 (+ 37955us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:44.817480 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:44.817611 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:44.818969 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:44.818988 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:44.819023 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:44.819226 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.819238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.827887 (+ 8649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.827893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.829753 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.829771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.831736 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.831743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.831940 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.832010 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 04:12:44.832102 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 04:12:44.832267 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37836,"thread_start_us":6581,"threads_started":1}
I20260807 04:12:44.833262 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb968841889421bb33f65bb628a1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:44.834026 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.779761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.18.193.1:38007)
0807 04:12:44.810800 (+ 31039us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.810813 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.818265 (+ 7452us) server_negotiation.cc:407] Connection header received
0807 04:12:44.818388 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.818396 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.818712 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.821451 (+ 2739us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:44.821755 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.825966 (+ 4211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.830007 (+ 4041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.831517 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.832273 (+ 756us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.832585 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.832692 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.832875 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30843,"thread_start_us":30739,"threads_started":1}
I20260807 04:12:44.838223 451 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb968841889421bb33f65bb628a1b70 (DEFAULT_TABLE table=test-table [id=fd14bcb9c9aa4dbf8ae0f577d4a95662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:12:44.839834 451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb968841889421bb33f65bb628a1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:44.840286 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.819394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33297 (local address 127.0.0.1:39416)
0807 04:12:44.819550 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:44.819571 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:44.819673 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:44.822419 (+ 2746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:44.822424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:44.822449 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:44.822773 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.822784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.825229 (+ 2445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.825236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.827262 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.827284 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.839476 (+ 12192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.839484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.839631 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.839657 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 04:12:44.839782 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 04:12:44.839940 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 04:12:44.840378 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.819534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.18.193.3:33297)
0807 04:12:44.821433 (+ 1899us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.821442 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.821474 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:12:44.821567 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.821574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.822201 (+ 627us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.823043 (+ 842us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:44.823352 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.825045 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.828263 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.838468 (+ 10205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.838644 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.838836 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.839839 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.840014 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1729,"thread_start_us":1573,"threads_started":1}
I20260807 04:12:44.844338 738 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb968841889421bb33f65bb628a1b70 (DEFAULT_TABLE table=test-table [id=fd14bcb9c9aa4dbf8ae0f577d4a95662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:12:44.845990 738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb968841889421bb33f65bb628a1b70. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:12:44 dist-test-slave-8n49 krb5kdc[31707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for HTTP/127.18.193.20@KRBTEST.COM
I20260807 04:12:44.869680 828 tablet_bootstrap.cc:492] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Bootstrap starting.
I20260807 04:12:44.875571 826 tablet_bootstrap.cc:492] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Bootstrap starting.
I20260807 04:12:44.879310 826 tablet_bootstrap.cc:654] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:44.880033 828 tablet_bootstrap.cc:654] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:44.882447 827 tablet_bootstrap.cc:492] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Bootstrap starting.
I20260807 04:12:44.883764 828 log.cc:824] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:44.885948 827 tablet_bootstrap.cc:654] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:44.886353 826 log.cc:824] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:44.888072 827 log.cc:824] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:44.895247 826 tablet_bootstrap.cc:492] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: No bootstrap required, opened a new log
I20260807 04:12:44.895717 826 ts_tablet_manager.cc:1397] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.000s
I20260807 04:12:44.904732 827 tablet_bootstrap.cc:492] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: No bootstrap required, opened a new log
I20260807 04:12:44.905217 827 ts_tablet_manager.cc:1397] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.013s
I20260807 04:12:44.909323 826 raft_consensus.cc:348] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.909828 826 raft_consensus.cc:374] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:44.909938 826 raft_consensus.cc:729] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0f490ee8b0431fa1614d9ac1f51553, State: Initialized, Role: FOLLOWER
I20260807 04:12:44.910838 826 consensus_queue.cc:260] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.916499 828 tablet_bootstrap.cc:492] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: No bootstrap required, opened a new log
I20260807 04:12:44.916863 828 ts_tablet_manager.cc:1397] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.007s
I20260807 04:12:44.916864 827 raft_consensus.cc:348] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.917330 827 raft_consensus.cc:374] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:44.917573 827 raft_consensus.cc:729] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70496a1b310f43ce91f83669accc2073, State: Initialized, Role: FOLLOWER
I20260807 04:12:44.918326 827 consensus_queue.cc:260] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.940514 827 ts_tablet_manager.cc:1428] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Time spent starting tablet: real 0.035s user 0.017s sys 0.002s
I20260807 04:12:44.944090 805 heartbeater.cc:503] Master 127.18.193.62:33175 was elected leader, sending a full tablet report...
I20260807 04:12:44.947963 828 raft_consensus.cc:348] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.948522 828 raft_consensus.cc:374] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:44.948657 828 raft_consensus.cc:729] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3443a22037da4e39b7baa1b70cd8b1d0, State: Initialized, Role: FOLLOWER
I20260807 04:12:44.953709 828 consensus_queue.cc:260] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.957376 666 heartbeater.cc:503] Master 127.18.193.62:33175 was elected leader, sending a full tablet report...
I20260807 04:12:44.958163 826 ts_tablet_manager.cc:1428] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Time spent starting tablet: real 0.062s user 0.031s sys 0.026s
I20260807 04:12:44.963753 519 heartbeater.cc:503] Master 127.18.193.62:33175 was elected leader, sending a full tablet report...
I20260807 04:12:44.963698 834 raft_consensus.cc:482] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:12:44.964404 834 raft_consensus.cc:504] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.966972 834 leader_election.cc:288] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0f490ee8b0431fa1614d9ac1f51553 (127.18.193.2:40671), 70496a1b310f43ce91f83669accc2073 (127.18.193.3:33297)
I20260807 04:12:44.960183 828 ts_tablet_manager.cc:1428] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Time spent starting tablet: real 0.043s user 0.013s sys 0.004s
I20260807 04:12:44.985921 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.967849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:51871 (local address 127.18.193.2:40671)
0807 04:12:44.968444 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.968451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.977917 (+ 9466us) server_negotiation.cc:407] Connection header received
0807 04:12:44.978250 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.978257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.978359 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.978432 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:44.979301 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.980637 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.983386 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.984624 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.984768 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.984846 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.985494 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.985639 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422}
I20260807 04:12:44.986811 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.967287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:40671 (local address 127.18.193.1:51871)
0807 04:12:44.977775 (+ 10488us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:44.977797 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:44.978059 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:44.978651 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:44.978656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:44.978683 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:44.979096 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.979111 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.980855 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.980861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.983179 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.983197 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.985074 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.985080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.985280 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.985316 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:12:44.986239 (+ 923us) client_negotiation.cc:240] Negotiation successful
0807 04:12:44.986385 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10338,"thread_start_us":13279,"threads_started":1}
I20260807 04:12:44.993309 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb968841889421bb33f65bb628a1b70" candidate_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" is_pre_election: true
I20260807 04:12:44.993947 619 raft_consensus.cc:2454] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3443a22037da4e39b7baa1b70cd8b1d0 in term 0.
I20260807 04:12:44.994051 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.981368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:33201 (local address 127.18.193.3:33297)
0807 04:12:44.981970 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:44.981977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:44.987249 (+ 5272us) server_negotiation.cc:407] Connection header received
0807 04:12:44.987570 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:44.987576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:44.987656 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:44.987718 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:44.988345 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.989442 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.991612 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.992868 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.993006 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.993096 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:44.993653 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 04:12:44.993788 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430}
I20260807 04:12:44.994735 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:44.981581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33297 (local address 127.18.193.1:33201)
0807 04:12:44.987153 (+ 5572us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:44.987174 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:44.987385 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:44.987960 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:44.987964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:44.987990 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:44.988167 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.988178 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.989622 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.989627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.991416 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:44.991441 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:44.993245 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:44.993251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:44.993449 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:44.993491 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 04:12:44.994296 (+ 805us) client_negotiation.cc:240] Negotiation successful
0807 04:12:44.994439 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5465,"thread_start_us":10486,"threads_started":1}
I20260807 04:12:44.994827 401 leader_election.cc:302] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3443a22037da4e39b7baa1b70cd8b1d0, fa0f490ee8b0431fa1614d9ac1f51553; no voters:
I20260807 04:12:44.995283 834 raft_consensus.cc:2781] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:12:44.995426 834 raft_consensus.cc:482] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:12:44.995519 834 raft_consensus.cc:3037] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:44.998840 834 raft_consensus.cc:504] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:44.999619 834 leader_election.cc:288] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [CANDIDATE]: Term 1 election: Requested vote from peers fa0f490ee8b0431fa1614d9ac1f51553 (127.18.193.2:40671), 70496a1b310f43ce91f83669accc2073 (127.18.193.3:33297)
I20260807 04:12:45.000100 759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb968841889421bb33f65bb628a1b70" candidate_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70496a1b310f43ce91f83669accc2073" is_pre_election: true
I20260807 04:12:45.000355 619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb968841889421bb33f65bb628a1b70" candidate_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0f490ee8b0431fa1614d9ac1f51553"
I20260807 04:12:45.000624 619 raft_consensus.cc:3037] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:45.000746 759 raft_consensus.cc:2454] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3443a22037da4e39b7baa1b70cd8b1d0 in term 0.
I20260807 04:12:45.001842 759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb968841889421bb33f65bb628a1b70" candidate_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70496a1b310f43ce91f83669accc2073"
I20260807 04:12:45.002041 759 raft_consensus.cc:3037] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:45.003770 619 raft_consensus.cc:2454] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3443a22037da4e39b7baa1b70cd8b1d0 in term 1.
I20260807 04:12:45.004405 401 leader_election.cc:302] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3443a22037da4e39b7baa1b70cd8b1d0, fa0f490ee8b0431fa1614d9ac1f51553; no voters:
I20260807 04:12:45.005405 759 raft_consensus.cc:2454] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3443a22037da4e39b7baa1b70cd8b1d0 in term 1.
I20260807 04:12:45.005431 834 raft_consensus.cc:2781] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:12:45.005693 834 raft_consensus.cc:686] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3443a22037da4e39b7baa1b70cd8b1d0, State: Running, Role: LEADER
I20260807 04:12:45.006222 834 consensus_queue.cc:237] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } }
I20260807 04:12:45.021462 312 catalog_manager.cc:5495] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3443a22037da4e39b7baa1b70cd8b1d0 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3443a22037da4e39b7baa1b70cd8b1d0" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38007 } health_report { overall_health: HEALTHY } } }
W20260807 04:12:45.062089 520 tablet.cc:2369] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:12:45.095021 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:45.082112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.18.193.1:38007)
0807 04:12:45.082328 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:45.082335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:45.086982 (+ 4647us) server_negotiation.cc:407] Connection header received
0807 04:12:45.087126 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:45.087132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:45.087238 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:45.087393 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:12:45.087981 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.089143 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.091311 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.091974 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.092108 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:45.093041 (+ 933us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:12:45.093804 (+ 763us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:12:45.093965 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:45.094641 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 04:12:45.094731 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:12:45.117477 759 raft_consensus.cc:1261] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 1 FOLLOWER]: Refusing update from remote peer 3443a22037da4e39b7baa1b70cd8b1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:12:45.118885 619 raft_consensus.cc:1261] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 1 FOLLOWER]: Refusing update from remote peer 3443a22037da4e39b7baa1b70cd8b1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:12:45.120074 834 consensus_queue.cc:1035] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0f490ee8b0431fa1614d9ac1f51553" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 40671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:12:45.121233 834 consensus_queue.cc:1035] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70496a1b310f43ce91f83669accc2073" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:12:45.155866 840 mvcc.cc:205] Tried to move back new op lower bound from 7315767153105874944 to 7315767152670642176. Current Snapshot: MvccSnapshot[applied={T|T < 7315767153105874944}]
I20260807 04:12:45.175575 842 mvcc.cc:205] Tried to move back new op lower bound from 7315767153105874944 to 7315767152670642176. Current Snapshot: MvccSnapshot[applied={T|T < 7315767153105874944}]
I20260807 04:12:45.212817 312 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:12:45.213131 312 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:12:45.217751 312 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 288d718b558f48cea3b1e54d5d0586c5: Sending DeleteTablet for 3 replicas of tablet 2bb968841889421bb33f65bb628a1b70
I20260807 04:12:45.218981 599 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb968841889421bb33f65bb628a1b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:12:45 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:57816
I20260807 04:12:45.220397 738 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb968841889421bb33f65bb628a1b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:12:45 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:39416
I20260807 04:12:45.227054 451 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb968841889421bb33f65bb628a1b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:12:45 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:39824
I20260807 04:12:45.229899 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 352
I20260807 04:12:45.237713 855 tablet_replica.cc:330] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: stopping tablet replica
I20260807 04:12:45.238804 855 raft_consensus.cc:2229] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:12:45.239627 855 raft_consensus.cc:2258] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:12:45.242134 855 ts_tablet_manager.cc:1905] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:12:45.248281 854 tablet_replica.cc:330] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: stopping tablet replica
I20260807 04:12:45.248811 854 raft_consensus.cc:2229] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:12:45.249385 854 raft_consensus.cc:2258] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:12:45.250281 852 tablet_replica.cc:330] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: stopping tablet replica
I20260807 04:12:45.250694 852 raft_consensus.cc:2229] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:12:45.251204 852 raft_consensus.cc:2258] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:12:45.255635 852 ts_tablet_manager.cc:1905] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:12:45.255635 854 ts_tablet_manager.cc:1905] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:12:45.257292 855 ts_tablet_manager.cc:1918] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:12:45.257503 855 log.cc:1196] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/2bb968841889421bb33f65bb628a1b70
I20260807 04:12:45.258116 855 ts_tablet_manager.cc:1939] T 2bb968841889421bb33f65bb628a1b70 P 3443a22037da4e39b7baa1b70cd8b1d0: Deleting consensus metadata
I20260807 04:12:45.258956 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:45.231368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.18.193.1:38007)
0807 04:12:45.232010 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:45.232018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:45.238170 (+ 6152us) server_negotiation.cc:407] Connection header received
0807 04:12:45.238430 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:45.238436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:45.238529 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:45.238640 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:45.240493 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.241716 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.244379 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.245122 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.245255 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:45.250813 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:45.250829 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:45.250846 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:45.250900 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:45.255119 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:45.256080 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:45.256089 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:45.256093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:45.256278 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:45.256769 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:45.256774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:45.256776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:45.257257 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:45.257599 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:45.258263 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 04:12:45.258575 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481}
I20260807 04:12:45.260087 32764 catalog_manager.cc:4841] TS 3443a22037da4e39b7baa1b70cd8b1d0 (127.18.193.1:38007): tablet 2bb968841889421bb33f65bb628a1b70 (table test-table [id=fd14bcb9c9aa4dbf8ae0f577d4a95662]) successfully deleted
I20260807 04:12:45.260715 513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:45.263268 854 ts_tablet_manager.cc:1918] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:12:45.263461 854 log.cc:1196] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/2bb968841889421bb33f65bb628a1b70
I20260807 04:12:45.264005 854 ts_tablet_manager.cc:1939] T 2bb968841889421bb33f65bb628a1b70 P 70496a1b310f43ce91f83669accc2073: Deleting consensus metadata
I20260807 04:12:45.265919 32763 catalog_manager.cc:4841] TS 70496a1b310f43ce91f83669accc2073 (127.18.193.3:33297): tablet 2bb968841889421bb33f65bb628a1b70 (table test-table [id=fd14bcb9c9aa4dbf8ae0f577d4a95662]) successfully deleted
I20260807 04:12:45.269294 852 ts_tablet_manager.cc:1918] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:12:45.269603 852 log.cc:1196] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/2bb968841889421bb33f65bb628a1b70
I20260807 04:12:45.270267 852 ts_tablet_manager.cc:1939] T 2bb968841889421bb33f65bb628a1b70 P fa0f490ee8b0431fa1614d9ac1f51553: Deleting consensus metadata
I20260807 04:12:45.271780 32763 catalog_manager.cc:4841] TS fa0f490ee8b0431fa1614d9ac1f51553 (127.18.193.2:40671): tablet 2bb968841889421bb33f65bb628a1b70 (table test-table [id=fd14bcb9c9aa4dbf8ae0f577d4a95662]) successfully deleted
I20260807 04:12:45.388916 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 352
I20260807 04:12:45.407850 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 523
I20260807 04:12:45.421190 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:45.408702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57824 (local address 127.18.193.2:40671)
0807 04:12:45.408871 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:45.408875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:45.408894 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:12:45.409049 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:45.409053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:45.409116 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:45.409203 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:45.410528 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.411232 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.412411 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.412838 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.412930 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:45.415345 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:45.415358 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:45.415373 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:45.415406 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:45.418600 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:45.419231 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:45.419238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:45.419240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:45.419370 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:45.419708 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:45.419711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:45.419713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:45.420091 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:45.420304 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:45.420543 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 04:12:45.420793 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:12:45.421866 659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:45.511727 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 523
I20260807 04:12:45.529881 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 671
I20260807 04:12:45.543607 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:45.530693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.18.193.3:33297)
0807 04:12:45.531238 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:45.531242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:45.531259 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:12:45.531315 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:45.531318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:45.531378 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:45.531480 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:45.532824 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.533603 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.534952 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:45.535350 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:45.535446 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:45.538411 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:45.538424 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:45.538441 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:45.538473 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:45.541208 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:45.541842 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:45.541849 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:45.541851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:45.541991 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:45.542286 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:45.542289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:45.542291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:45.542611 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:45.542836 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:45.543054 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 04:12:45.543214 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":310,"threads_started":1}
I20260807 04:12:45.544288 800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:45.637699 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 671
I20260807 04:12:45.656119 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 32746
I20260807 04:12:45.657447 344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:45.772822 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 32746
I20260807 04:12:45.797242 19204 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 04:12:46.142642 19204 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 04:12:46.142905 19204 mini_ranger.cc:66] Stopping Ranger...
I20260807 04:12:46.488533 19204 mini_ranger.cc:68] Stopped Ranger
2026-08-07 04:12:46.488 UTC [31739] LOG: received smart shutdown request
2026-08-07 04:12:46.491 UTC [31739] LOG: background worker "logical replication launcher" (PID 31749) exited with exit code 1
2026-08-07 04:12:46.492 UTC [31743] LOG: shutting down
2026-08-07 04:12:46.672 UTC [31739] LOG: database system is shut down
2026-08-07T04:12:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54466 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:12:46 dist-test-slave-8n49 krb5kdc[875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:12:46 dist-test-slave-8n49 krb5kdc[875](info): set up 2 sockets
Aug 07 04:12:46 dist-test-slave-8n49 krb5kdc[875](info): commencing operation
krb5kdc: starting...
W20260807 04:12:47.966555 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:12:48 dist-test-slave-8n49 krb5kdc[875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:12:48.238233 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34251
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:34251,127.18.193.61:39307,127.18.193.60:38101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:48.579682 888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:48.579988 888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:48.580077 888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:48.587481 888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:48.587594 888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:48.587642 888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:48.587702 888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:48.587745 888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:48.597884 888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:34251,127.18.193.61:39307,127.18.193.60:38101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:48.599201 888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:48.600482 888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:48.612696 896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:48.612632 893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:48.612762 894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:48.614090 888 server_base.cc:1034] running on GCE node
I20260807 04:12:48.614949 888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:48.616438 888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:48.617681 888 hybrid_clock.cc:648] HybridClock initialized: now 1786075968617633 us; error 44 us; skew 500 ppm
I20260807 04:12:48.618039 888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:48 dist-test-slave-8n49 krb5kdc[875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:48.621236 888 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:12:48.622767 888 webserver.cc:460] Webserver started at http://127.18.193.62:39529/ using document root <none> and password file <none>
I20260807 04:12:48.623513 888 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:48.623653 888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:48.623968 888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:48.627072 888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "22a1d0d32eaa43b99ecf38a6502401c6"
format_stamp: "Formatted at 2026-08-07 04:12:48 on dist-test-slave-8n49"
server_key: "907e3fc6b91bfcd562b31b74fc105a94"
server_key_iv: "b9c88f6513192b9005495232c237862d"
server_key_version: "encryptionkey@0"
I20260807 04:12:48.627972 888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "22a1d0d32eaa43b99ecf38a6502401c6"
format_stamp: "Formatted at 2026-08-07 04:12:48 on dist-test-slave-8n49"
server_key: "907e3fc6b91bfcd562b31b74fc105a94"
server_key_iv: "b9c88f6513192b9005495232c237862d"
server_key_version: "encryptionkey@0"
I20260807 04:12:48.633488 888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 04:12:48.637185 904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:48.638614 888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:12:48.638810 888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "22a1d0d32eaa43b99ecf38a6502401c6"
format_stamp: "Formatted at 2026-08-07 04:12:48 on dist-test-slave-8n49"
server_key: "907e3fc6b91bfcd562b31b74fc105a94"
server_key_iv: "b9c88f6513192b9005495232c237862d"
server_key_version: "encryptionkey@0"
I20260807 04:12:48.639026 888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:48.657565 888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:48.661839 888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:48.662130 888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:48.681031 888 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:34251
I20260807 04:12:48.681118 958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:34251 every 8 connection(s)
I20260807 04:12:48.682441 888 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:12:48.684799 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 888
I20260807 04:12:48.685021 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:12:48.685672 19204 external_mini_cluster.cc:1428] Setting key ba5415ec9331d6ff4899315ed63a70be
I20260807 04:12:48.686311 960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:48.692996 960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:12:48 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.193.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.61@KRBTEST.COM" created.
I20260807 04:12:48.715510 965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.697616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.18.193.62:34251)
0807 04:12:48.698473 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:48.698481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:48.698533 (+ 52us) server_negotiation.cc:407] Connection header received
0807 04:12:48.699682 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:48.699688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:48.699906 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:48.700103 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:48.701399 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:48.702620 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:48.703994 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:48.704535 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:48.704789 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:48.708151 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:48.708159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:48.708170 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:48.708197 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:48.711082 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:48.711889 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:48.711898 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:48.711902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:48.712061 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:48.712452 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:48.712460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:48.712463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:48.713028 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:48.713230 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:48.713746 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 04:12:48.714085 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":607,"thread_start_us":426,"threads_started":1}
I20260807 04:12:48.715535 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.696270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34251 (local address 127.0.0.1:48812)
0807 04:12:48.697099 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.697143 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:48.699305 (+ 2162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:48.700120 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:48.700127 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:48.700855 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:48.701195 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:48.701212 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:48.702779 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:48.702784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:48.703840 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:48.703852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:48.704821 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:48.704827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:48.705033 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:48.705765 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:48.705787 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:48.707932 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:48.711289 (+ 3357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:48.711303 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:48.711316 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:48.711764 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:48.712169 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:48.712176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:48.712179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:48.712343 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:48.713168 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:48.713178 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:48.713384 (+ 206us) client_negotiation.cc:769] Sending connection context
0807 04:12:48.713627 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 04:12:48.714194 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":132,"thread_start_us":362,"threads_started":1}
I20260807 04:12:48.718645 960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:12:48.719811 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.719389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40274)
0807 04:12:48.719572 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.719696 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 04:12:48.720098 905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
I20260807 04:12:48.721779 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.721388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40280)
0807 04:12:48.721591 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.721656 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260807 04:12:48.722092 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:39307: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab.
Entry for principal kudu/127.18.193.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab.
I20260807 04:12:48.745141 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:39307
--webserver_interface=127.18.193.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab
--principal=kudu/127.18.193.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:34251,127.18.193.61:39307,127.18.193.60:38101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:12:48.771519 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } attempt: 1
I20260807 04:12:48.772830 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.772303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40286)
0807 04:12:48.772511 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.772688 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 04:12:48.773805 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.773462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40296)
0807 04:12:48.773615 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.773683 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 04:12:48.774017 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:39307: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
I20260807 04:12:48.842314 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } attempt: 2
I20260807 04:12:48.843578 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.843100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40298)
0807 04:12:48.843346 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.843438 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 04:12:48.844683 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.844194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40308)
0807 04:12:48.844373 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.844484 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 04:12:48.844893 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:39307: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
I20260807 04:12:48.936244 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } attempt: 3
I20260807 04:12:48.937574 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.937118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40316)
0807 04:12:48.937330 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.937439 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 04:12:48.938700 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:48.938358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40326)
0807 04:12:48.938496 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:48.938570 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 04:12:48.938972 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:39307: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
I20260807 04:12:49.082245 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } attempt: 4
I20260807 04:12:49.083534 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.082992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40340)
0807 04:12:49.083312 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.083397 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260807 04:12:49.084810 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.084300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40350)
0807 04:12:49.084473 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.084611 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 04:12:49.085129 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:39307: Network error: Client connection negotiation failed: client connection to 127.18.193.61:39307: connect: Connection refused (error 111)
W20260807 04:12:49.091917 968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:49.092216 968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:49.092393 968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:49.099752 968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:49.099854 968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:49.099982 968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:49.100082 968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:49.100150 968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:49.110276 968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:34251,127.18.193.61:39307,127.18.193.60:38101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab
--principal=kudu/127.18.193.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.193.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:49.111759 968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:49.113121 968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:49.125321 973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:49.125272 976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:49.125413 974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:49.126065 968 server_base.cc:1034] running on GCE node
I20260807 04:12:49.126848 968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:49.128281 968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:49.129503 968 hybrid_clock.cc:648] HybridClock initialized: now 1786075969129460 us; error 52 us; skew 500 ppm
I20260807 04:12:49.129861 968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:49.133266 968 init.cc:377] Logged in from keytab as kudu/127.18.193.61@KRBTEST.COM (short username kudu)
I20260807 04:12:49.134684 968 webserver.cc:460] Webserver started at http://127.18.193.61:38413/ using document root <none> and password file <none>
I20260807 04:12:49.135334 968 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:49.135428 968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:49.135673 968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:49.138700 968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data/instance:
uuid: "0c788eaf00a740948c72ff8cfd0fc91c"
format_stamp: "Formatted at 2026-08-07 04:12:49 on dist-test-slave-8n49"
server_key: "71991df26059813680fc8ed82b37f253"
server_key_iv: "4b1986416de43170db63b93f93d6071b"
server_key_version: "encryptionkey@0"
I20260807 04:12:49.139473 968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/wal/instance:
uuid: "0c788eaf00a740948c72ff8cfd0fc91c"
format_stamp: "Formatted at 2026-08-07 04:12:49 on dist-test-slave-8n49"
server_key: "71991df26059813680fc8ed82b37f253"
server_key_iv: "4b1986416de43170db63b93f93d6071b"
server_key_version: "encryptionkey@0"
I20260807 04:12:49.144776 968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 04:12:49.148247 983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:49.149649 968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:12:49.149834 968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/wal
uuid: "0c788eaf00a740948c72ff8cfd0fc91c"
format_stamp: "Formatted at 2026-08-07 04:12:49 on dist-test-slave-8n49"
server_key: "71991df26059813680fc8ed82b37f253"
server_key_iv: "4b1986416de43170db63b93f93d6071b"
server_key_version: "encryptionkey@0"
I20260807 04:12:49.150039 968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:49.179265 968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:49.183799 968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:49.184106 968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:49.203176 968 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.61:39307
I20260807 04:12:49.203208 1035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.61:39307 every 8 connection(s)
I20260807 04:12:49.204594 968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
I20260807 04:12:49.208390 1036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:49.213320 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 968
I20260807 04:12:49.213539 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-1/wal/instance
I20260807 04:12:49.214008 19204 external_mini_cluster.cc:1428] Setting key 5bb337d84a73ab1caad6a4f2011dd879
I20260807 04:12:49.215031 1036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:49.237612 1038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.217727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.18.193.62:34251)
0807 04:12:49.218299 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.218308 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.219256 (+ 948us) server_negotiation.cc:407] Connection header received
0807 04:12:49.221218 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.221227 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.221321 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.221472 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.222977 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.223753 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.226333 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.226785 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.226895 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.231274 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.231279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.231283 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.231321 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.234225 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.235131 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.235135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.235137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.235258 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.235681 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.235686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.235693 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.235973 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.236153 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.236951 (+ 798us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.237204 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":262,"threads_started":1}
WARNING: no policy specified for kudu/127.18.193.60@KRBTEST.COM; defaulting to no policy
I20260807 04:12:49.238555 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.218275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34251 (local address 127.0.0.1:48814)
0807 04:12:49.219070 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.219116 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.220945 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.221660 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.221668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.222173 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.222773 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.222790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.223930 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.223934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.226161 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.226180 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.227036 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.227046 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.227362 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.228149 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.228184 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.231018 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.234382 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.234391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.234410 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.234997 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.235370 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.235375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.235379 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.235563 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.236122 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.236130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.236523 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.236854 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.237400 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":334,"threads_started":1}
I20260807 04:12:49.240317 1036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.60@KRBTEST.COM" created.
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:12:49.257252 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.241190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40364)
0807 04:12:49.241430 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.241453 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.241576 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.242728 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.242732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.243153 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.243295 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.243303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.245340 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.245344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.246172 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.246183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.247429 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.247434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.247591 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.248207 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.248224 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.250741 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.254726 (+ 3985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.254734 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.254738 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.255105 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.255506 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.255511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.255514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.255646 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.256267 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.256271 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.256417 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.256578 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.256859 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 04:12:49.257459 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.241434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40364 (local address 127.18.193.61:39307)
0807 04:12:49.242200 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.242209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.242242 (+ 33us) server_negotiation.cc:407] Connection header received
0807 04:12:49.242296 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.242304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.242554 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.242708 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.243455 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.245203 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.246312 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.247278 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.247423 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.250961 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.250977 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.250995 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.251034 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.254608 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.255236 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.255245 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.255253 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.255367 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.255748 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.255756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.255759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.256128 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.256348 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.256802 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.257059 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":394,"threads_started":1}
I20260807 04:12:49.259977 1036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:12:49.261107 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.260674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38036)
0807 04:12:49.260820 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.260950 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:12:49.261340 984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
I20260807 04:12:49.262928 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.262609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38042)
0807 04:12:49.262748 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.262829 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 04:12:49.263167 1036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab.
Entry for principal kudu/127.18.193.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab.
I20260807 04:12:49.275475 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:38101
--webserver_interface=127.18.193.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab
--principal=kudu/127.18.193.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:34251,127.18.193.61:39307,127.18.193.60:38101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:12:49.312517 1036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 1
I20260807 04:12:49.313890 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.313397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38048)
0807 04:12:49.313613 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.313720 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 04:12:49.315121 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.314755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38058)
0807 04:12:49.314897 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.315001 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 04:12:49.315410 1036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
I20260807 04:12:49.383662 1036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 2
I20260807 04:12:49.384384 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } attempt: 5
I20260807 04:12:49.385007 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.384410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38066)
0807 04:12:49.384669 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.384788 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 04:12:49.386178 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.385737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38078)
0807 04:12:49.385951 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.386034 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260807 04:12:49.386436 1036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:12:49.398715 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.385039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40374)
0807 04:12:49.385297 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.385315 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.385448 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.385855 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.385860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.386271 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.386506 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.386523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.387939 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.387943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.388903 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.388915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.390006 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.390011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.390146 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.390788 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.390806 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.392798 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.396406 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.396410 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.396413 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.396822 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.397251 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.397255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.397258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.397396 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.398019 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.398022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.398152 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.398261 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.398479 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260807 04:12:49.398977 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.385105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40374 (local address 127.18.193.61:39307)
0807 04:12:49.385349 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.385357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.385392 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:12:49.385581 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.385585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.385702 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.385822 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.386672 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.387794 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.389041 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.389818 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.389972 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.392986 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.392993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.392997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.393037 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.396262 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.396944 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.396949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.396953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.397105 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.397523 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.397528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.397531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.397898 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.398045 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.398412 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.398681 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260807 04:12:49.400151 960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:12:49.401168 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.400797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38080)
0807 04:12:49.400961 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.401041 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 04:12:49.402041 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.401735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38088)
0807 04:12:49.401870 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.401951 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 04:12:49.402258 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
I20260807 04:12:49.453483 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 1
I20260807 04:12:49.454825 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.454231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38100)
0807 04:12:49.454484 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.454610 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 04:12:49.455993 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.455591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38110)
0807 04:12:49.455748 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.455839 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:12:49.456254 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
I20260807 04:12:49.477691 1036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 3
I20260807 04:12:49.479100 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.478553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38112)
0807 04:12:49.478824 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.478928 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 04:12:49.480383 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.479912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38114)
0807 04:12:49.480181 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.480263 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260807 04:12:49.480731 1036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
I20260807 04:12:49.524526 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 2
I20260807 04:12:49.526002 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.525340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38124)
0807 04:12:49.525614 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.525754 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 04:12:49.527240 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.526777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38136)
0807 04:12:49.526987 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.527089 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 04:12:49.527494 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
W20260807 04:12:49.622922 1046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:49.623224 1046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:49.623426 1046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 04:12:49.624007 1036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 4
I20260807 04:12:49.625240 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.624748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38144)
0807 04:12:49.624994 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.625088 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 04:12:49.626238 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.625917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38152)
0807 04:12:49.626063 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.626136 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:12:49.626466 1036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
W20260807 04:12:49.631156 1046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:49.631258 1046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:49.631353 1046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:49.631424 1046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:49.631501 1046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:49.633731 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 3
I20260807 04:12:49.634903 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.634376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38166)
0807 04:12:49.634596 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.634700 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 04:12:49.636070 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.635668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38168)
0807 04:12:49.635842 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.635936 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 04:12:49.636338 960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:38101: Network error: Client connection negotiation failed: client connection to 127.18.193.60:38101: connect: Connection refused (error 111)
I20260807 04:12:49.642481 1046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:34251,127.18.193.61:39307,127.18.193.60:38101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab
--principal=kudu/127.18.193.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:38101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.193.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:49.643932 1046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:49.645448 1046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:49.657855 1052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:49.657812 1053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:49.657774 1055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:49.658703 1046 server_base.cc:1034] running on GCE node
I20260807 04:12:49.659534 1046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:49.661130 1046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:49.662340 1046 hybrid_clock.cc:648] HybridClock initialized: now 1786075969662302 us; error 39 us; skew 500 ppm
I20260807 04:12:49.662755 1046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:49.666311 1046 init.cc:377] Logged in from keytab as kudu/127.18.193.60@KRBTEST.COM (short username kudu)
I20260807 04:12:49.667937 1046 webserver.cc:460] Webserver started at http://127.18.193.60:45947/ using document root <none> and password file <none>
I20260807 04:12:49.668717 1046 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:49.668833 1046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:49.669191 1046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:49.672436 1046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data/instance:
uuid: "4cac841c82dc44a7acb2183d2f3a1582"
format_stamp: "Formatted at 2026-08-07 04:12:49 on dist-test-slave-8n49"
server_key: "cb9a60a2ad9b1612e65d35b2443ea839"
server_key_iv: "4f250ebeaedf0ff5b9f25a0c691a9580"
server_key_version: "encryptionkey@0"
I20260807 04:12:49.673383 1046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/wal/instance:
uuid: "4cac841c82dc44a7acb2183d2f3a1582"
format_stamp: "Formatted at 2026-08-07 04:12:49 on dist-test-slave-8n49"
server_key: "cb9a60a2ad9b1612e65d35b2443ea839"
server_key_iv: "4f250ebeaedf0ff5b9f25a0c691a9580"
server_key_version: "encryptionkey@0"
I20260807 04:12:49.678730 1046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:12:49.682436 1064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:49.683816 1046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:12:49.683996 1046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/wal
uuid: "4cac841c82dc44a7acb2183d2f3a1582"
format_stamp: "Formatted at 2026-08-07 04:12:49 on dist-test-slave-8n49"
server_key: "cb9a60a2ad9b1612e65d35b2443ea839"
server_key_iv: "4f250ebeaedf0ff5b9f25a0c691a9580"
server_key_version: "encryptionkey@0"
I20260807 04:12:49.684191 1046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:49.717890 1046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:49.722723 1046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:49.723058 1046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:49.744123 1046 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.60:38101
I20260807 04:12:49.744163 1118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.60:38101 every 8 connection(s)
I20260807 04:12:49.745729 1046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
I20260807 04:12:49.749693 1119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:49.753105 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1046
I20260807 04:12:49.753311 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786075749366013-19204-0/minicluster-data/master-2/wal/instance
I20260807 04:12:49.753944 19204 external_mini_cluster.cc:1428] Setting key e1b04a8887b13c38cc771f986e148213
I20260807 04:12:49.758217 1119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:49.777209 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.761059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.18.193.62:34251)
0807 04:12:49.761589 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.761599 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.762165 (+ 566us) server_negotiation.cc:407] Connection header received
0807 04:12:49.762567 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.762573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.762700 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.762995 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.764081 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.764976 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.767083 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.767745 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.767909 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.771298 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.771302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.771306 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.771348 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.773933 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.774624 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.774630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.774633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.774802 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.775234 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.775239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.775241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.775538 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.775658 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.776469 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.776757 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260807 04:12:49.778282 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.755985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38180 (local address 127.18.193.60:38101)
0807 04:12:49.756870 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.756882 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.756923 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:12:49.758600 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.758622 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.759028 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.759405 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.760993 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.762910 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.764629 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.765480 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.765669 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.770936 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.770947 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.770966 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.771006 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.773985 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.774736 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.774743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.774745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.774927 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.775340 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.775343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.775345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.776071 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.776378 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.776780 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.776981 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":663,"spinlock_wait_cycles":22272,"thread_start_us":392,"threads_started":1}
I20260807 04:12:49.778318 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.761121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34251 (local address 127.0.0.1:48826)
0807 04:12:49.761995 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.762055 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.762223 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.763065 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.763074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.763649 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.763910 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.763918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.765134 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.765140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.766918 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.766936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.767989 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.767994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.768126 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.768784 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.768806 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.771084 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.774073 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.774086 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.774101 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.774502 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.774909 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.774913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.774918 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.775120 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.775670 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.775679 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.776062 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.776352 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.776871 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":56,"thread_start_us":463,"threads_started":1}
I20260807 04:12:49.779937 1119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:49.794623 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.780788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:39307 (local address 127.0.0.1:40376)
0807 04:12:49.780928 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.780949 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.781045 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.781420 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.781425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.781853 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.782039 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.782051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.783500 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.783504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.784826 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.784841 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.785796 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.785801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.785951 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.786534 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.786551 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.789159 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.792255 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.792258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.792261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.792692 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.793146 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.793149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.793152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.793281 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.793887 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.793893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.794087 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.794200 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.794386 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 04:12:49.795158 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.780801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.18.193.61:39307)
0807 04:12:49.781003 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.781010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.781032 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:12:49.781149 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.781154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.781240 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.781328 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.782227 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.783332 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.784969 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.785628 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.785780 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.789342 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.789347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.789350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.789383 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.792130 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.792822 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.792831 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.792836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.793007 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.793400 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.793406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.793411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.793760 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.793904 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.794360 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.794584 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:12:49.796090 1119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:12:49.798969 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.781915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.18.193.62:34251)
0807 04:12:49.782082 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.782087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.782106 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:12:49.782297 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.782301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.782368 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.782443 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.783890 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.784969 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.786625 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.787286 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.787416 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.792222 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.792228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.792234 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.792273 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.795609 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.796359 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.796365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.796369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.796516 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.796983 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.796990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.796996 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.797351 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.797614 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.798423 (+ 809us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.798665 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:12:49.809864 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.796846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38190)
0807 04:12:49.797038 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.797067 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.797159 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.797465 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.797468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.797771 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.797900 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.797907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.799067 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.799074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.800017 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.800030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.800928 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.800933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.801161 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.801702 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.801718 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.803996 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.807455 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.807458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.807461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.807797 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.808210 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.808213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.808215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.808331 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.808877 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.808882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.809010 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.809158 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.809459 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 04:12:49.809929 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.797052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.18.193.60:38101)
0807 04:12:49.797213 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.797220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.797235 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:12:49.797280 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.797284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.797349 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.797450 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.798014 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.798915 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.800164 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.800781 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.800895 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.804172 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.804177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.804180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.804207 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.807327 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.807922 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.807928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.807932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.808099 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.808447 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.808452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.808455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.808769 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.808914 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.809304 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.809619 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":56}
I20260807 04:12:49.813593 960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 4
I20260807 04:12:49.816459 1119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: Bootstrap starting.
I20260807 04:12:49.816529 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.800501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.18.193.61:39307)
0807 04:12:49.800669 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.800673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.800715 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:12:49.800903 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.800908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.800974 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.801077 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.802229 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.803141 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.804787 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.805287 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.805380 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.810305 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.810309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.810312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.810341 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.813587 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.814460 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.814466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.814470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.814623 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.815090 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.815094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.815096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.815346 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.815644 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.815963 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.816201 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 04:12:49.820776 1119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: Neither blocks nor log segments found. Creating new log.
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075968, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
I20260807 04:12:49.822137 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:12:49.822224 1119 log.cc:824] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:49.825877 1119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: No bootstrap required, opened a new log
I20260807 04:12:49.831362 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.814464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38194 (local address 127.18.193.60:38101)
0807 04:12:49.814631 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.814635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.814648 (+ 13us) server_negotiation.cc:407] Connection header received
0807 04:12:49.814705 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.814710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.814783 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.814890 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.815751 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.816899 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.818336 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.818863 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.818994 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.825409 (+ 6415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.825431 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.825437 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.825481 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.828471 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.829263 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.829269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.829273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.829418 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.829823 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.829828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.829835 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.830179 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.830341 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.830717 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.830941 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:12:49.831357 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.814185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38194)
0807 04:12:49.814354 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.814374 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.814480 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.814955 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.814960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.815409 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.815612 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.815627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.817048 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.817054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.818204 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.818217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.819020 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.819026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.819216 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.819817 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.819846 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.825181 (+ 5335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.828705 (+ 3524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.828711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.828715 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.829135 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.829537 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.829542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.829546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.829705 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.830302 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.830307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.830476 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.830619 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.830919 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 04:12:49.835922 960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6: Bootstrap starting.
I20260807 04:12:49.840005 960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:49.840515 1119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:49.841104 1119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:49.841253 1119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cac841c82dc44a7acb2183d2f3a1582, State: Initialized, Role: FOLLOWER
I20260807 04:12:49.841629 960 log.cc:824] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:49.842185 1119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:49.846508 1133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:49.847044 1133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:12:49.848100 1119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:12:49.848263 960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6: No bootstrap required, opened a new log
I20260807 04:12:49.860725 960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:49.861272 960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:49.861379 960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a1d0d32eaa43b99ecf38a6502401c6, State: Initialized, Role: FOLLOWER
I20260807 04:12:49.862222 960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:49.872164 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.839136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.18.193.62:34251)
0807 04:12:49.839294 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.839300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.845291 (+ 5991us) server_negotiation.cc:407] Connection header received
0807 04:12:49.847528 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.847533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.847614 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.847686 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.850146 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.851338 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.854421 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.855447 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.855595 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.862867 (+ 7272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.862874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.862878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.862940 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.867905 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.868840 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.868845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.868849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.869039 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.869509 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.869516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.869519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.869921 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.871340 (+ 1419us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.871484 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.871723 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 04:12:49.877854 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.838953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.18.193.61:39307)
0807 04:12:49.839579 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.839585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.845795 (+ 6210us) server_negotiation.cc:407] Connection header received
0807 04:12:49.846311 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.846319 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.846412 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.846521 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.849500 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.851017 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.854945 (+ 3928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.855670 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.855774 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.861727 (+ 5953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.861736 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.861742 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.861785 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.869816 (+ 8031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.875264 (+ 5448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.875270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.875273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.875455 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.875939 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.875944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.875950 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.876322 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.876622 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.877207 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.877474 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485}
I20260807 04:12:49.880090 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.846103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.18.193.60:38101)
0807 04:12:49.848952 (+ 2849us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.848959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.858475 (+ 9516us) server_negotiation.cc:407] Connection header received
0807 04:12:49.858628 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.858637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.858757 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.858871 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.861183 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.862285 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.864656 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.865355 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.865466 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.872270 (+ 6804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.872276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.872282 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.872325 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.876313 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.877171 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.877176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.877179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.877317 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.877732 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.877736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.877739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.878121 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.878365 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.879481 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.879722 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2712}
I20260807 04:12:49.880838 1138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:49.881608 1138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:12:49.882654 960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 04:12:49.892644 1146 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:12:49.892813 1146 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:12:49.887207 1046 master_runner.cc:186] Error getting master registration for 127.18.193.62:34251: OK, instance_id { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" instance_seqno: 1786075968675458 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:12:49.896339 1046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:12:49.904070 888 master_runner.cc:186] Error getting master registration for 127.18.193.61:39307: OK, instance_id { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" instance_seqno: 1786075969197686 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:12:49.912333 888 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 04:12:49.915025 910 tablet.cc:2369] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:12:49.915230 1158 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:12:49.915352 1158 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:12:49.925792 1036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } attempt: 5
I20260807 04:12:49.931402 1138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:12:49.931716 1138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:49.933554 1138 leader_election.cc:288] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c788eaf00a740948c72ff8cfd0fc91c (127.18.193.61:39307), 4cac841c82dc44a7acb2183d2f3a1582 (127.18.193.60:38101)
I20260807 04:12:49.934902 1092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac841c82dc44a7acb2183d2f3a1582" is_pre_election: true
Aug 07 04:12:49 dist-test-slave-8n49 krb5kdc[875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
I20260807 04:12:49.935561 1092 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22a1d0d32eaa43b99ecf38a6502401c6 in term 0.
I20260807 04:12:49.936277 905 leader_election.cc:302] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a1d0d32eaa43b99ecf38a6502401c6, 4cac841c82dc44a7acb2183d2f3a1582; no voters:
I20260807 04:12:49.937436 1138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:12:49.937604 1138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:12:49.937724 1138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:49.941256 1138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:49.946677 1138 leader_election.cc:288] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [CANDIDATE]: Term 1 election: Requested vote from peers 0c788eaf00a740948c72ff8cfd0fc91c (127.18.193.61:39307), 4cac841c82dc44a7acb2183d2f3a1582 (127.18.193.60:38101)
I20260807 04:12:49.946676 1092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cac841c82dc44a7acb2183d2f3a1582"
I20260807 04:12:49.946923 1092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:49.947207 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.926409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:38101 (local address 127.0.0.1:38210)
0807 04:12:49.926693 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:49.926714 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:49.926843 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:49.927262 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:49.927267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:49.927815 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:49.928091 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.928106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.929624 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.929630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.931090 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:49.931107 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.932482 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.932489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:49.932773 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.933511 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:49.933539 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:49.936693 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:49.942739 (+ 6046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.942746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.942752 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.943376 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.943858 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:49.943863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:49.943866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:49.944046 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:49.944803 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:49.944809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:49.944973 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 04:12:49.945137 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 04:12:49.945438 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260807 04:12:49.948422 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:49.926480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38210 (local address 127.18.193.60:38101)
0807 04:12:49.926702 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:49.926709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:49.926770 (+ 61us) server_negotiation.cc:407] Connection header received
0807 04:12:49.926993 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:49.926998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:49.927093 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:49.927199 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:49.928275 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.929448 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.931252 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:49.931952 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:49.932093 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:49.936918 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:49.936926 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:49.936931 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:49.936971 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:49.942565 (+ 5594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.943523 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.943529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.943532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.943706 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:49.944175 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:49.944180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:49.944183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:49.944590 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:49.944738 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:49.946231 (+ 1493us) server_negotiation.cc:299] Negotiation successful
0807 04:12:49.946968 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 04:12:49.955174 1036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c: Bootstrap starting.
I20260807 04:12:49.958971 1092 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22a1d0d32eaa43b99ecf38a6502401c6 in term 1.
I20260807 04:12:49.960907 905 leader_election.cc:302] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a1d0d32eaa43b99ecf38a6502401c6, 4cac841c82dc44a7acb2183d2f3a1582; no voters:
I20260807 04:12:49.961283 1138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:12:49.968314 1036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:49.973510 1036 log.cc:824] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c: Log is configured to *not* fsync() on all Append() calls
W20260807 04:12:49.975415 1070 tablet.cc:2369] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:12:49.979066 1138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [term 1 LEADER]: Becoming Leader. State: Replica: 22a1d0d32eaa43b99ecf38a6502401c6, State: Running, Role: LEADER
I20260807 04:12:49.979718 1138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:49.983246 1160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22a1d0d32eaa43b99ecf38a6502401c6. Latest consensus state: current_term: 1 leader_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:49.983762 1160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:49.987664 1036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c: No bootstrap required, opened a new log
I20260807 04:12:49.988369 1163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:12:49.993093 1163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:12:50.000032 1036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:50.000514 1036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:50.000734 1036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c788eaf00a740948c72ff8cfd0fc91c, State: Initialized, Role: FOLLOWER
I20260807 04:12:50.001582 1036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } }
I20260807 04:12:50.008754 1164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:50.011442 1164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:12:50.011356 1009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c788eaf00a740948c72ff8cfd0fc91c"
I20260807 04:12:50.011301 1036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:12:50.011901 1009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:50.012650 1010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" is_pre_election: true
I20260807 04:12:50.021611 1092 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [term 1 FOLLOWER]: Refusing update from remote peer 22a1d0d32eaa43b99ecf38a6502401c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:12:50.021605 1009 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22a1d0d32eaa43b99ecf38a6502401c6 in term 1.
I20260807 04:12:50.022867 1160 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:12:50.025415 1009 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [term 1 FOLLOWER]: Refusing update from remote peer 22a1d0d32eaa43b99ecf38a6502401c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:12:50.026391 1160 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:12:50.049258 1164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22a1d0d32eaa43b99ecf38a6502401c6. Latest consensus state: current_term: 1 leader_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:50.049647 1164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:12:50.054308 1164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:50.054617 1164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:12:50.059005 1133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22a1d0d32eaa43b99ecf38a6502401c6. Latest consensus state: current_term: 1 leader_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:50.059454 1133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:12:50.063134 1160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:50.063428 1160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:50.063793 1160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:50.063999 1160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:50.064463 1133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a1d0d32eaa43b99ecf38a6502401c6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34251 } } peers { permanent_uuid: "0c788eaf00a740948c72ff8cfd0fc91c" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 39307 } } peers { permanent_uuid: "4cac841c82dc44a7acb2183d2f3a1582" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 38101 } } }
I20260807 04:12:50.064740 1133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:12:50.065656 1163 catalog_manager.cc:1347] Generated new cluster ID: 685fd4f406bb440192e21862a123bc86
I20260807 04:12:50.065781 1163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:12:50.095345 1187 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 04:12:50.146819 1163 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:12:50.148950 1163 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:12:50.158707 1163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22a1d0d32eaa43b99ecf38a6502401c6: Generated new TSK 0
I20260807 04:12:50.159835 1163 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:12:50.894851 1146 catalog_manager.cc:1259] Loaded cluster ID: 685fd4f406bb440192e21862a123bc86
I20260807 04:12:50.894974 1146 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: loading cluster ID for follower catalog manager: success
I20260807 04:12:50.897526 1146 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: acquiring CA information for follower catalog manager: success
I20260807 04:12:50.899379 1146 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4cac841c82dc44a7acb2183d2f3a1582: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:12:51.100005 1187 catalog_manager.cc:1259] Loaded cluster ID: 685fd4f406bb440192e21862a123bc86
I20260807 04:12:51.100159 1187 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c: loading cluster ID for follower catalog manager: success
I20260807 04:12:51.102798 1187 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c: acquiring CA information for follower catalog manager: success
I20260807 04:12:51.104803 1187 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c788eaf00a740948c72ff8cfd0fc91c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:12:51.257221 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 888
I20260807 04:12:51.258358 954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:51.373857 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 888
I20260807 04:12:51.407106 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 968
I20260807 04:12:51.408421 1031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:51.524421 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 968
I20260807 04:12:51.546577 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1046
I20260807 04:12:51.547785 1113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:51.655478 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1046
2026-08-07T04:12:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4844 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:12:51 dist-test-slave-8n49 krb5kdc[1200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:12:51 dist-test-slave-8n49 krb5kdc[1200](info): set up 2 sockets
Aug 07 04:12:51 dist-test-slave-8n49 krb5kdc[1200](info): commencing operation
krb5kdc: starting...
W20260807 04:12:52.803121 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:12:52 dist-test-slave-8n49 krb5kdc[1200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:12:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:12:53.084882 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44809
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:44809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:53.426811 1213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:53.427170 1213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:53.427338 1213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:53.434603 1213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:53.434762 1213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:53.434870 1213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:53.434975 1213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:53.435058 1213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:53.444890 1213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:44809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:53.446249 1213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:53.447580 1213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:53.459508 1218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:53.459486 1219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:53.459560 1221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:53.460767 1213 server_base.cc:1034] running on GCE node
I20260807 04:12:53.461558 1213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:53.463040 1213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:53.464286 1213 hybrid_clock.cc:648] HybridClock initialized: now 1786075973464200 us; error 88 us; skew 500 ppm
I20260807 04:12:53.464694 1213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:53 dist-test-slave-8n49 krb5kdc[1200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075973, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:53.473033 1213 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:12:53.474646 1213 webserver.cc:460] Webserver started at http://127.18.193.62:44383/ using document root <none> and password file <none>
I20260807 04:12:53.475370 1213 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:53.475517 1213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:53.475831 1213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:53.479020 1213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "499ba2126acd4be9a07f78d3c61728b9"
format_stamp: "Formatted at 2026-08-07 04:12:53 on dist-test-slave-8n49"
server_key: "5fcba676531c46606f7ed5bd54b05edb"
server_key_iv: "f7782fd8245e0118f7bcf4c10f7567cf"
server_key_version: "encryptionkey@0"
I20260807 04:12:53.480132 1213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "499ba2126acd4be9a07f78d3c61728b9"
format_stamp: "Formatted at 2026-08-07 04:12:53 on dist-test-slave-8n49"
server_key: "5fcba676531c46606f7ed5bd54b05edb"
server_key_iv: "f7782fd8245e0118f7bcf4c10f7567cf"
server_key_version: "encryptionkey@0"
I20260807 04:12:53.485446 1213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:12:53.489001 1228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:53.490340 1213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:12:53.490517 1213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "499ba2126acd4be9a07f78d3c61728b9"
format_stamp: "Formatted at 2026-08-07 04:12:53 on dist-test-slave-8n49"
server_key: "5fcba676531c46606f7ed5bd54b05edb"
server_key_iv: "f7782fd8245e0118f7bcf4c10f7567cf"
server_key_version: "encryptionkey@0"
I20260807 04:12:53.490720 1213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:53.506125 1213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:53.510541 1213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:53.510838 1213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:53.529960 1213 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:44809
I20260807 04:12:53.529986 1281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:44809 every 8 connection(s)
I20260807 04:12:53.531411 1213 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:12:53.535211 1282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:53.541198 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1213
I20260807 04:12:53.541463 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:12:53.542297 19204 external_mini_cluster.cc:1428] Setting key 75e18c5c79366c4a4554ff977e9a74f1
I20260807 04:12:53.546608 1282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9: Bootstrap starting.
I20260807 04:12:53.550355 1282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:53.551455 1282 log.cc:824] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:53.554503 1282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9: No bootstrap required, opened a new log
Aug 07 04:12:53 dist-test-slave-8n49 krb5kdc[1200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:53.563015 1282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499ba2126acd4be9a07f78d3c61728b9" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44809 } }
I20260807 04:12:53.563383 1282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:53.563526 1282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499ba2126acd4be9a07f78d3c61728b9, State: Initialized, Role: FOLLOWER
I20260807 04:12:53.564159 1282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499ba2126acd4be9a07f78d3c61728b9" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44809 } }
I20260807 04:12:53.564424 1282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:12:53.564613 1282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:12:53.564814 1282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:53.566358 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:53.544578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.18.193.62:44809)
0807 04:12:53.545454 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:53.545465 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:53.545505 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:12:53.547283 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:53.547306 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:53.547678 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:53.547999 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:53.549926 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:53.551297 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:53.553030 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:53.553737 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:53.553932 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:53.558851 (+ 4919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:53.558859 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:53.558873 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:53.558909 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:53.562108 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:53.562847 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:53.562854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:53.562860 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:53.563031 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:53.563434 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:53.563438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:53.563440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:53.564054 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:53.564392 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:53.564868 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 04:12:53.565310 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":405,"threads_started":1}
I20260807 04:12:53.567354 1282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499ba2126acd4be9a07f78d3c61728b9" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44809 } }
I20260807 04:12:53.567878 1282 leader_election.cc:302] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 499ba2126acd4be9a07f78d3c61728b9; no voters:
I20260807 04:12:53.568517 1282 leader_election.cc:288] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:12:53.568675 1287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:12:53.570096 1287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [term 1 LEADER]: Becoming Leader. State: Replica: 499ba2126acd4be9a07f78d3c61728b9, State: Running, Role: LEADER
I20260807 04:12:53.570585 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:12:53.570819 1282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:12:53.570745 1287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499ba2126acd4be9a07f78d3c61728b9" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44809 } }
I20260807 04:12:53.573884 1288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "499ba2126acd4be9a07f78d3c61728b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499ba2126acd4be9a07f78d3c61728b9" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44809 } } }
I20260807 04:12:53.573774 1289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 499ba2126acd4be9a07f78d3c61728b9. Latest consensus state: current_term: 1 leader_uuid: "499ba2126acd4be9a07f78d3c61728b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499ba2126acd4be9a07f78d3c61728b9" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44809 } } }
I20260807 04:12:53.574352 1288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:53.574381 1289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:53.575361 1293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:12:53.583263 1293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:12:53.592229 1293 catalog_manager.cc:1347] Generated new cluster ID: 4e0c9b5cfe704fd5a481d6f2300665d9
I20260807 04:12:53.592352 1293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:12:53.614153 1293 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:12:53.615502 1293 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:12:53.627942 1293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 499ba2126acd4be9a07f78d3c61728b9: Generated new TSK 0
I20260807 04:12:53.629029 1293 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:12:53.684079 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1213
I20260807 04:12:53.685309 1277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:53.785055 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1213
2026-08-07T04:12:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2125 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:12:53 dist-test-slave-8n49 krb5kdc[1311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:12:53 dist-test-slave-8n49 krb5kdc[1311](info): set up 2 sockets
Aug 07 04:12:53 dist-test-slave-8n49 krb5kdc[1311](info): commencing operation
krb5kdc: starting...
W20260807 04:12:54.930435 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:12:55 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:12:55.203969 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38421
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:38421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:55.541447 1325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:55.541743 1325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:55.541903 1325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:55.549078 1325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:55.549190 1325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:55.549280 1325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:55.549386 1325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:55.549465 1325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:55.559476 1325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:38421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:55.560820 1325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:55.562107 1325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:55.574213 1333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:55.574278 1334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:55.574244 1336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:55.575389 1325 server_base.cc:1034] running on GCE node
I20260807 04:12:55.576246 1325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:55.577766 1325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:55.579005 1325 hybrid_clock.cc:648] HybridClock initialized: now 1786075975578964 us; error 69 us; skew 500 ppm
I20260807 04:12:55.579351 1325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:55 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075975, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:55.582538 1325 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:12:55.583995 1325 webserver.cc:460] Webserver started at http://127.18.193.62:43513/ using document root <none> and password file <none>
I20260807 04:12:55.584738 1325 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:55.584888 1325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:55.585189 1325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:55.588155 1325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "75c2b73495e84fb08447fcab183b72b7"
format_stamp: "Formatted at 2026-08-07 04:12:55 on dist-test-slave-8n49"
server_key: "ee9e594e20e03e2ba2291ff3bb79e5f5"
server_key_iv: "e1d64b79f14fa5952e3aae3cbc33bb70"
server_key_version: "encryptionkey@0"
I20260807 04:12:55.589035 1325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "75c2b73495e84fb08447fcab183b72b7"
format_stamp: "Formatted at 2026-08-07 04:12:55 on dist-test-slave-8n49"
server_key: "ee9e594e20e03e2ba2291ff3bb79e5f5"
server_key_iv: "e1d64b79f14fa5952e3aae3cbc33bb70"
server_key_version: "encryptionkey@0"
I20260807 04:12:55.594159 1325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:12:55.597620 1343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:55.599035 1325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:12:55.599222 1325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "75c2b73495e84fb08447fcab183b72b7"
format_stamp: "Formatted at 2026-08-07 04:12:55 on dist-test-slave-8n49"
server_key: "ee9e594e20e03e2ba2291ff3bb79e5f5"
server_key_iv: "e1d64b79f14fa5952e3aae3cbc33bb70"
server_key_version: "encryptionkey@0"
I20260807 04:12:55.599467 1325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:55.628353 1325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:55.632694 1325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:55.632997 1325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:55.654326 1325 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:38421
I20260807 04:12:55.654368 1397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:38421 every 8 connection(s)
I20260807 04:12:55.655742 1325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:12:55.659574 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:55.663156 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1325
I20260807 04:12:55.663534 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:12:55.664302 19204 external_mini_cluster.cc:1428] Setting key c4b473640aca1401880335d99153cfdf
I20260807 04:12:55.671048 1398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Bootstrap starting.
I20260807 04:12:55.674753 1398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:55.676026 1398 log.cc:824] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:55.679518 1398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: No bootstrap required, opened a new log
Aug 07 04:12:55 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:55.687713 1398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:55.688110 1398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:55.688249 1398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c2b73495e84fb08447fcab183b72b7, State: Initialized, Role: FOLLOWER
I20260807 04:12:55.688935 1398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:55.689183 1398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:12:55.689340 1398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:12:55.689465 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:55.666622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58352 (local address 127.18.193.62:38421)
0807 04:12:55.667491 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:55.667502 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:55.667548 (+ 46us) server_negotiation.cc:407] Connection header received
0807 04:12:55.669603 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:55.669626 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:55.670038 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:55.670366 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:55.672440 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:55.673996 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:55.675922 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:55.676843 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:55.677068 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:55.681854 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:55.681865 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:55.681882 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:55.681920 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:55.685120 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:55.685819 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:55.685834 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:55.685840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:55.686030 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:55.686437 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:55.686443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:55.686446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:55.687027 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:55.687338 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:55.687776 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 04:12:55.688227 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":406,"threads_started":1}
I20260807 04:12:55.689534 1398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:55.692613 1398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:55.693147 1398 leader_election.cc:302] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75c2b73495e84fb08447fcab183b72b7; no voters:
I20260807 04:12:55.693709 1398 leader_election.cc:288] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:12:55.693853 1403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:12:55.694527 1403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 LEADER]: Becoming Leader. State: Replica: 75c2b73495e84fb08447fcab183b72b7, State: Running, Role: LEADER
I20260807 04:12:55.695042 1403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:55.695717 1398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:12:55.698541 1405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c2b73495e84fb08447fcab183b72b7. Latest consensus state: current_term: 1 leader_uuid: "75c2b73495e84fb08447fcab183b72b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } } }
I20260807 04:12:55.698997 1405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:55.699136 1404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75c2b73495e84fb08447fcab183b72b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } } }
I20260807 04:12:55.699651 1404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:55.704011 1411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:12:55.709240 1411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:12:55.718968 1411 catalog_manager.cc:1347] Generated new cluster ID: 26b2d60b0a5844aaae80c996c0acb9cf
I20260807 04:12:55.719105 1411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:12:55.734236 1411 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:12:55.736011 1411 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:12:55.744473 1411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Generated new TSK 0
I20260807 04:12:55.745321 1411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:12:55.828859 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38421
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:56.200380 1427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:56.200733 1427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:56.200925 1427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:56.208519 1427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:56.208703 1427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:56.208900 1427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:12:56.218777 1427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:56.220090 1427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:56.221472 1427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:56.234405 1435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:56.234453 1434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:56.234489 1437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:56.235311 1427 server_base.cc:1034] running on GCE node
I20260807 04:12:56.235994 1427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:56.236997 1427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:56.238221 1427 hybrid_clock.cc:648] HybridClock initialized: now 1786075976238176 us; error 52 us; skew 500 ppm
I20260807 04:12:56.238632 1427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:56 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:56.241849 1427 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:12:56.243270 1427 webserver.cc:460] Webserver started at http://127.18.193.1:34231/ using document root <none> and password file <none>
I20260807 04:12:56.243979 1427 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:56.244128 1427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:56.244477 1427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:56.247522 1427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "e8ebc115dced738463c268bc5fed9b9a"
server_key_iv: "d42645dc0440827bd9d5b532b4b98f28"
server_key_version: "encryptionkey@0"
I20260807 04:12:56.248327 1427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "e8ebc115dced738463c268bc5fed9b9a"
server_key_iv: "d42645dc0440827bd9d5b532b4b98f28"
server_key_version: "encryptionkey@0"
I20260807 04:12:56.253610 1427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 04:12:56.257077 1445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:56.258466 1427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:12:56.258667 1427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "e8ebc115dced738463c268bc5fed9b9a"
server_key_iv: "d42645dc0440827bd9d5b532b4b98f28"
server_key_version: "encryptionkey@0"
I20260807 04:12:56.258934 1427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:56.278575 1427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:56.282912 1427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:56.283169 1427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:56.284088 1427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:56.285506 1427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:56.285585 1427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:56.285676 1427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:56.285722 1427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:56.320173 1427 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:38059
I20260807 04:12:56.320212 1565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:38059 every 8 connection(s)
I20260807 04:12:56.321516 1427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:12:56.330422 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1427
I20260807 04:12:56.330644 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:12:56.331318 19204 external_mini_cluster.cc:1428] Setting key c2c1eb3ff6c759ae49e8429675c7b1b0
Aug 07 04:12:56 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:56.342095 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:56.324123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:56777 (local address 127.18.193.62:38421)
0807 04:12:56.324658 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:56.324662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:56.325600 (+ 938us) server_negotiation.cc:407] Connection header received
0807 04:12:56.327396 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:56.327402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:56.327493 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:56.327611 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:56.329162 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.329929 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.331596 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.332080 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.332232 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:56.336278 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:56.336283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:56.336286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:56.336318 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:56.338968 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:56.339701 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:56.339712 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:56.339715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:56.339830 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:56.340262 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:56.340267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:56.340270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:56.340535 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:56.340742 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:56.341466 (+ 724us) server_negotiation.cc:299] Negotiation successful
0807 04:12:56.341711 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260807 04:12:56.343436 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:56.324758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:56777)
0807 04:12:56.325449 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:56.325485 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:56.327135 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:56.327853 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:56.327861 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:56.328365 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:56.328985 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:56.329003 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.330101 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.330105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:56.331433 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:56.331449 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.332327 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.332336 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:56.332653 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:56.333399 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:56.333436 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:56.336055 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:56.339119 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:56.339128 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:56.339145 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:56.339575 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:56.339940 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:56.339945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:56.339948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:56.340145 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:56.340727 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:56.340735 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:56.341084 (+ 349us) client_negotiation.cc:769] Sending connection context
0807 04:12:56.341357 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 04:12:56.341907 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":269,"threads_started":1}
I20260807 04:12:56.345201 1566 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38421
I20260807 04:12:56.345515 1566 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:56.346297 1566 heartbeater.cc:511] Master 127.18.193.62:38421 requested a full tablet report, sending...
I20260807 04:12:56.348677 1362 ts_manager.cc:194] Registered new tserver with Master: bb2b5cbcb93c4710a03b3dccf1cc8535 (127.18.193.1:38059)
I20260807 04:12:56.350941 1362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:56777
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:12:56.393131 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38421
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:56.726732 1572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:56.727120 1572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:56.727231 1572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:56.735005 1572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:56.735138 1572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:56.735260 1572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:12:56.746248 1572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:56.747619 1572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:56.749188 1572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:56.763198 1579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:56.763545 1578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:56.763419 1581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:56.764669 1572 server_base.cc:1034] running on GCE node
I20260807 04:12:56.765498 1572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:56.766501 1572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:56.767736 1572 hybrid_clock.cc:648] HybridClock initialized: now 1786075976767672 us; error 54 us; skew 500 ppm
I20260807 04:12:56.768078 1572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:56 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:56.771324 1572 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:12:56.772987 1572 webserver.cc:460] Webserver started at http://127.18.193.2:38507/ using document root <none> and password file <none>
I20260807 04:12:56.773689 1572 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:56.773835 1572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:56.774143 1572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:56.777441 1572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "79f3613ddcb24cd6959496592242f197"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "2f254b99a2e1d9ed0e2fe1c1a0200944"
server_key_iv: "d2bef970fffc5f5ce988dc170ca561ed"
server_key_version: "encryptionkey@0"
I20260807 04:12:56.778237 1572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "79f3613ddcb24cd6959496592242f197"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "2f254b99a2e1d9ed0e2fe1c1a0200944"
server_key_iv: "d2bef970fffc5f5ce988dc170ca561ed"
server_key_version: "encryptionkey@0"
I20260807 04:12:56.783440 1572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:12:56.787228 1588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:56.788812 1572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:12:56.789000 1572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "79f3613ddcb24cd6959496592242f197"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "2f254b99a2e1d9ed0e2fe1c1a0200944"
server_key_iv: "d2bef970fffc5f5ce988dc170ca561ed"
server_key_version: "encryptionkey@0"
I20260807 04:12:56.789219 1572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:56.803110 1572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:56.807287 1572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:56.807551 1572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:56.808503 1572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:56.809881 1572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:56.809957 1572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:56.810042 1572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:56.810093 1572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:56.845130 1572 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:36801
I20260807 04:12:56.845180 1710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:36801 every 8 connection(s)
I20260807 04:12:56.846455 1572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:12:56.849526 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1572
I20260807 04:12:56.849737 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:12:56.850358 19204 external_mini_cluster.cc:1428] Setting key 050f61b388cbf3c72405cbeb8a0a236e
Aug 07 04:12:56 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:56.869453 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:56.849063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:34249 (local address 127.18.193.62:38421)
0807 04:12:56.849560 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:56.849565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:56.850781 (+ 1216us) server_negotiation.cc:407] Connection header received
0807 04:12:56.852952 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:56.852958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:56.853046 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:56.853177 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:56.854717 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.855461 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.857110 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.857552 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.857692 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:56.861857 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:56.861866 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:56.861873 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:56.861920 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:56.865894 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:56.866702 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:56.866711 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:56.866717 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:56.866881 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:56.867357 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:56.867366 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:56.867372 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:56.867779 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:56.867936 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:56.868658 (+ 722us) server_negotiation.cc:299] Negotiation successful
0807 04:12:56.868923 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":263,"threads_started":1}
I20260807 04:12:56.870107 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:56.849705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:34249)
0807 04:12:56.850614 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:56.850654 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:56.852673 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:56.853423 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:56.853432 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:56.853951 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:56.854528 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:56.854545 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.855618 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.855626 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:56.856952 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:56.856965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:56.857806 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:56.857814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:56.858051 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:56.858788 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:56.858812 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:56.861551 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:56.866068 (+ 4517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:56.866075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:56.866088 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:56.866550 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:56.867018 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:56.867022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:56.867024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:56.867219 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:56.867904 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:56.867909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:56.868241 (+ 332us) client_negotiation.cc:769] Sending connection context
0807 04:12:56.868469 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 04:12:56.868950 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":357,"threads_started":1}
I20260807 04:12:56.871627 1711 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38421
I20260807 04:12:56.871973 1711 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:56.872912 1711 heartbeater.cc:511] Master 127.18.193.62:38421 requested a full tablet report, sending...
I20260807 04:12:56.874552 1362 ts_manager.cc:194] Registered new tserver with Master: 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801)
I20260807 04:12:56.875396 1362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:34249
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:12:56.916618 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38421
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:57.258440 1717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:57.258762 1717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:57.258930 1717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:57.266702 1717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:57.266856 1717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:57.267043 1717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:12:57.277462 1717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:57.278771 1717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:57.280050 1717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:57.293452 1723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:57.293486 1725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:57.293532 1722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:57.304340 1717 server_base.cc:1034] running on GCE node
I20260807 04:12:57.305207 1717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:57.306319 1717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:57.307574 1717 hybrid_clock.cc:648] HybridClock initialized: now 1786075977307534 us; error 34 us; skew 500 ppm
I20260807 04:12:57.307919 1717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:57 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:57.311303 1717 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:12:57.313140 1717 webserver.cc:460] Webserver started at http://127.18.193.3:38149/ using document root <none> and password file <none>
I20260807 04:12:57.313943 1717 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:57.314100 1717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:57.314437 1717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:12:57.317685 1717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "f48d92e63a154a61996e4776f3d024b9"
format_stamp: "Formatted at 2026-08-07 04:12:57 on dist-test-slave-8n49"
server_key: "c8b06f4a22f5e61b80bc5a5a95c40f6d"
server_key_iv: "deba32761a499dda6cefde262bc36e81"
server_key_version: "encryptionkey@0"
I20260807 04:12:57.318562 1717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "f48d92e63a154a61996e4776f3d024b9"
format_stamp: "Formatted at 2026-08-07 04:12:57 on dist-test-slave-8n49"
server_key: "c8b06f4a22f5e61b80bc5a5a95c40f6d"
server_key_iv: "deba32761a499dda6cefde262bc36e81"
server_key_version: "encryptionkey@0"
I20260807 04:12:57.325546 1717 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 04:12:57.329422 1733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:57.330756 1717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:12:57.330893 1717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "f48d92e63a154a61996e4776f3d024b9"
format_stamp: "Formatted at 2026-08-07 04:12:57 on dist-test-slave-8n49"
server_key: "c8b06f4a22f5e61b80bc5a5a95c40f6d"
server_key_iv: "deba32761a499dda6cefde262bc36e81"
server_key_version: "encryptionkey@0"
I20260807 04:12:57.331074 1717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:57.354281 1566 heartbeater.cc:503] Master 127.18.193.62:38421 was elected leader, sending a full tablet report...
I20260807 04:12:57.366513 1717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:57.371778 1717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:57.372146 1717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:57.373556 1717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:57.375351 1717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:57.375445 1717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:57.375578 1717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:57.375640 1717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:57.410033 1717 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33851
I20260807 04:12:57.410089 1854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33851 every 8 connection(s)
I20260807 04:12:57.411346 1717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:12:57.414822 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1717
I20260807 04:12:57.415035 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:12:57.415627 19204 external_mini_cluster.cc:1428] Setting key e29a456008dfcc31aa967070bfee2547
Aug 07 04:12:57 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:57.433059 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.413935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:43681 (local address 127.18.193.62:38421)
0807 04:12:57.414470 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.414477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.415591 (+ 1114us) server_negotiation.cc:407] Connection header received
0807 04:12:57.418229 (+ 2638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.418235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.418321 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.418457 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:57.420263 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.421143 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.422723 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.423101 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.423199 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.426763 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:57.426768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:57.426771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:57.426804 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:57.429593 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:57.430500 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:57.430506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:57.430511 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:57.430644 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:57.431232 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:57.431237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:57.431240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:57.431546 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:57.431668 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.432519 (+ 851us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.432756 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":229,"threads_started":1}
I20260807 04:12:57.434047 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.414521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:43681)
0807 04:12:57.415378 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:57.415437 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:57.417950 (+ 2513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:57.418751 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:57.418766 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:57.419501 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:57.420099 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.420118 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.421304 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.421309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.422582 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.422594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.423348 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.423354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.423571 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.424210 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:57.424237 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:57.426529 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:57.429754 (+ 3225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:57.429764 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:57.429782 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:57.430371 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:57.430801 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:57.430804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:57.430807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:57.431096 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:57.431678 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:57.431687 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:57.432099 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 04:12:57.432388 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 04:12:57.432874 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":343,"threads_started":1}
I20260807 04:12:57.435647 1855 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38421
I20260807 04:12:57.435971 1855 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:57.436779 1855 heartbeater.cc:511] Master 127.18.193.62:38421 requested a full tablet report, sending...
I20260807 04:12:57.438226 1362 ts_manager.cc:194] Registered new tserver with Master: f48d92e63a154a61996e4776f3d024b9 (127.18.193.3:33851)
I20260807 04:12:57.439050 1362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:43681
I20260807 04:12:57.444252 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:12:57.461601 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.448999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.18.193.62:38421)
0807 04:12:57.449177 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.449181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.449709 (+ 528us) server_negotiation.cc:407] Connection header received
0807 04:12:57.449835 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.449839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.449906 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.450003 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:57.451404 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.452140 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.453643 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.454046 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.454153 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.456524 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:57.456530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:57.456533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:57.456625 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:57.459265 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:57.459882 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:57.459886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:57.459888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:57.460010 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:57.460329 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:57.460332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:57.460334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:57.460716 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:57.460900 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.461188 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.461354 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:12:57.466143 1362 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:12:57.473320 1362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:12:57.520330 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.486144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.18.193.3:33851)
0807 04:12:57.511485 (+ 25341us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.511493 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.511521 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:12:57.511613 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.511619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.511805 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.512252 (+ 447us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:57.512996 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.515096 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.518064 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.519126 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.519282 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.519528 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.519778 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.519940 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25181,"thread_start_us":25081,"threads_started":1}
I20260807 04:12:57.520596 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.485731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33851 (local address 127.0.0.1:60728)
0807 04:12:57.508857 (+ 23126us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:57.508901 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:57.509127 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:57.512033 (+ 2906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:57.512044 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:57.512100 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:57.512755 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.512768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.515241 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.515247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.517876 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.517890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.519318 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.519324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.519519 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.519573 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 04:12:57.519718 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 04:12:57.519918 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22871,"mutex_wait_us":106,"spinlock_wait_cycles":4992,"thread_start_us":23939,"threads_started":1}
I20260807 04:12:57.521354 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.486088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:54998)
0807 04:12:57.509931 (+ 23843us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:57.509948 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:57.510048 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:57.512039 (+ 1991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:57.512048 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:57.512123 (+ 75us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:57.513389 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.513403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.515160 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.515165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.518033 (+ 2868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.518049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.520249 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.520258 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.520688 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.520749 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 04:12:57.520832 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 04:12:57.520963 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23719,"mutex_wait_us":137,"spinlock_wait_cycles":16768,"thread_start_us":26322,"threads_started":1}
I20260807 04:12:57.521505 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.487100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.18.193.2:36801)
0807 04:12:57.507915 (+ 20815us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.507929 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.512881 (+ 4952us) server_negotiation.cc:407] Connection header received
0807 04:12:57.513140 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.513151 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.513398 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.513558 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:57.514549 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.515785 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.518400 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.519806 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.519946 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.520109 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.520957 (+ 848us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.521156 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20542,"thread_start_us":20845,"threads_started":1}
I20260807 04:12:57.522068 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.486845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.18.193.1:38059)
0807 04:12:57.511297 (+ 24452us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.511308 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.511341 (+ 33us) server_negotiation.cc:407] Connection header received
0807 04:12:57.511447 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.511457 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.511733 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.511967 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:57.513654 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.515015 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.518199 (+ 3184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.519683 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.519831 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.520075 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.521629 (+ 1554us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.521804 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24259,"thread_start_us":24095,"threads_started":1}
I20260807 04:12:57.522709 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.486551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:36801 (local address 127.0.0.1:58704)
0807 04:12:57.512751 (+ 26200us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:57.512766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:57.512911 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:57.513916 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:57.513921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:57.513951 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:57.514359 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.514372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.515894 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.515899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.518229 (+ 2330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.518242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.520315 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.520321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.520693 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.520759 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 04:12:57.521759 (+ 1000us) client_negotiation.cc:240] Negotiation successful
0807 04:12:57.522382 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26097,"mutex_wait_us":69,"thread_start_us":21867,"threads_started":1}
I20260807 04:12:57.524935 1784 tablet_service.cc:1467] Processing CreateTablet for tablet e8168a2652794101a74017b6bf718ba2 (DEFAULT_TABLE table=test-table [id=3b9c121c189b4825807b0683ee33faa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:12:57.526140 1784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8168a2652794101a74017b6bf718ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:57.526182 1640 tablet_service.cc:1467] Processing CreateTablet for tablet e8168a2652794101a74017b6bf718ba2 (DEFAULT_TABLE table=test-table [id=3b9c121c189b4825807b0683ee33faa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:12:57.526178 1495 tablet_service.cc:1467] Processing CreateTablet for tablet e8168a2652794101a74017b6bf718ba2 (DEFAULT_TABLE table=test-table [id=3b9c121c189b4825807b0683ee33faa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:12:57.527525 1495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8168a2652794101a74017b6bf718ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:57.527546 1640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8168a2652794101a74017b6bf718ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:12:57.537627 1875 tablet_bootstrap.cc:492] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Bootstrap starting.
I20260807 04:12:57.540910 1875 tablet_bootstrap.cc:654] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:57.542063 1875 log.cc:824] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:57.542615 1876 tablet_bootstrap.cc:492] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Bootstrap starting.
I20260807 04:12:57.544203 1879 tablet_bootstrap.cc:492] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Bootstrap starting.
I20260807 04:12:57.547670 1879 tablet_bootstrap.cc:654] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:57.548538 1876 tablet_bootstrap.cc:654] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Neither blocks nor log segments found. Creating new log.
I20260807 04:12:57.548843 1879 log.cc:824] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:57.549796 1876 log.cc:824] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:57.551540 1875 tablet_bootstrap.cc:492] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: No bootstrap required, opened a new log
I20260807 04:12:57.551872 1875 ts_tablet_manager.cc:1397] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260807 04:12:57.552845 1876 tablet_bootstrap.cc:492] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: No bootstrap required, opened a new log
I20260807 04:12:57.553046 1879 tablet_bootstrap.cc:492] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: No bootstrap required, opened a new log
I20260807 04:12:57.553126 1876 ts_tablet_manager.cc:1397] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260807 04:12:57.553279 1879 ts_tablet_manager.cc:1397] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260807 04:12:57.559406 1875 raft_consensus.cc:348] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.559808 1875 raft_consensus.cc:374] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:57.559901 1875 raft_consensus.cc:729] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f48d92e63a154a61996e4776f3d024b9, State: Initialized, Role: FOLLOWER
I20260807 04:12:57.560536 1875 consensus_queue.cc:260] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.561725 1855 heartbeater.cc:503] Master 127.18.193.62:38421 was elected leader, sending a full tablet report...
I20260807 04:12:57.562611 1875 ts_tablet_manager.cc:1428] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260807 04:12:57.563967 1876 raft_consensus.cc:348] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.563943 1879 raft_consensus.cc:348] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.564409 1876 raft_consensus.cc:374] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:57.564409 1879 raft_consensus.cc:374] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:12:57.564512 1876 raft_consensus.cc:729] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2b5cbcb93c4710a03b3dccf1cc8535, State: Initialized, Role: FOLLOWER
I20260807 04:12:57.564510 1879 raft_consensus.cc:729] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79f3613ddcb24cd6959496592242f197, State: Initialized, Role: FOLLOWER
I20260807 04:12:57.565161 1876 consensus_queue.cc:260] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.565161 1879 consensus_queue.cc:260] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.566808 1711 heartbeater.cc:503] Master 127.18.193.62:38421 was elected leader, sending a full tablet report...
I20260807 04:12:57.567087 1876 ts_tablet_manager.cc:1428] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 04:12:57.567397 1879 ts_tablet_manager.cc:1428] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
W20260807 04:12:57.573927 1567 tablet.cc:2369] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:12:57.598498 1712 tablet.cc:2369] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:12:57.663168 1856 tablet.cc:2369] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:12:57.766214 1883 raft_consensus.cc:482] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:12:57.766610 1883 raft_consensus.cc:504] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.769027 1883 leader_election.cc:288] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f48d92e63a154a61996e4776f3d024b9 (127.18.193.3:33851), 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801)
I20260807 04:12:57.776268 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.769637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:38267 (local address 127.18.193.3:33851)
0807 04:12:57.769880 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.769889 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.770864 (+ 975us) server_negotiation.cc:407] Connection header received
0807 04:12:57.771057 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.771062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.771146 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.771249 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:57.771843 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.772639 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.774583 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.775334 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.775527 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.775604 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.775879 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.776003 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 04:12:57.776358 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.769603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:47435 (local address 127.18.193.2:36801)
0807 04:12:57.769848 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.769857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.770634 (+ 777us) server_negotiation.cc:407] Connection header received
0807 04:12:57.770840 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.770846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.770926 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.771032 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:12:57.771730 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.772633 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.774648 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.775412 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.775617 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.775693 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.775966 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.776092 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:12:57.776407 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.769491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33851 (local address 127.18.193.1:38267)
0807 04:12:57.770735 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:57.770768 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:57.770898 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:57.771279 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:57.771283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:57.771306 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:57.771640 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.771676 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.772809 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.772814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.774430 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.774442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.775541 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.775547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.775714 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.775741 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 04:12:57.775842 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 04:12:57.775977 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"mutex_wait_us":63,"thread_start_us":536,"threads_started":1}
I20260807 04:12:57.776904 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.769505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:36801 (local address 127.18.193.1:47435)
0807 04:12:57.770494 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:57.770517 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:57.770687 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:57.771061 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:57.771066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:57.771102 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:12:57.771574 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.771585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.772797 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.772802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.774497 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:57.774512 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.775656 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.775663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:57.775815 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.775837 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 04:12:57.775940 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 04:12:57.776073 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":62,"thread_start_us":857,"threads_started":1}
I20260807 04:12:57.777485 1805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168a2652794101a74017b6bf718ba2" candidate_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f48d92e63a154a61996e4776f3d024b9" is_pre_election: true
I20260807 04:12:57.777988 1805 raft_consensus.cc:2454] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2b5cbcb93c4710a03b3dccf1cc8535 in term 0.
I20260807 04:12:57.778656 1449 leader_election.cc:302] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb2b5cbcb93c4710a03b3dccf1cc8535, f48d92e63a154a61996e4776f3d024b9; no voters:
I20260807 04:12:57.779072 1883 raft_consensus.cc:2781] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:12:57.779184 1883 raft_consensus.cc:482] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:12:57.779326 1883 raft_consensus.cc:3037] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:57.779201 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168a2652794101a74017b6bf718ba2" candidate_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f3613ddcb24cd6959496592242f197" is_pre_election: true
I20260807 04:12:57.779762 1662 raft_consensus.cc:2454] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2b5cbcb93c4710a03b3dccf1cc8535 in term 0.
I20260807 04:12:57.781874 1883 raft_consensus.cc:504] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.782553 1883 leader_election.cc:288] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [CANDIDATE]: Term 1 election: Requested vote from peers f48d92e63a154a61996e4776f3d024b9 (127.18.193.3:33851), 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801)
I20260807 04:12:57.782903 1805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168a2652794101a74017b6bf718ba2" candidate_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f48d92e63a154a61996e4776f3d024b9"
I20260807 04:12:57.783104 1805 raft_consensus.cc:3037] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:57.783034 1662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8168a2652794101a74017b6bf718ba2" candidate_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f3613ddcb24cd6959496592242f197"
I20260807 04:12:57.783263 1662 raft_consensus.cc:3037] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:12:57.785380 1805 raft_consensus.cc:2454] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2b5cbcb93c4710a03b3dccf1cc8535 in term 1.
I20260807 04:12:57.785538 1662 raft_consensus.cc:2454] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2b5cbcb93c4710a03b3dccf1cc8535 in term 1.
I20260807 04:12:57.785784 1449 leader_election.cc:302] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb2b5cbcb93c4710a03b3dccf1cc8535, f48d92e63a154a61996e4776f3d024b9; no voters:
I20260807 04:12:57.786077 1883 raft_consensus.cc:2781] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:12:57.786522 1883 raft_consensus.cc:686] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 LEADER]: Becoming Leader. State: Replica: bb2b5cbcb93c4710a03b3dccf1cc8535, State: Running, Role: LEADER
I20260807 04:12:57.786988 1883 consensus_queue.cc:237] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:12:57.791340 1362 catalog_manager.cc:5495] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb2b5cbcb93c4710a03b3dccf1cc8535 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } health_report { overall_health: UNKNOWN } } }
I20260807 04:12:57.822003 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.816448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.18.193.1:38059)
0807 04:12:57.816698 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.816703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.816723 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:12:57.816875 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.816879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.816974 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.817102 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:12:57.817482 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.818312 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.819758 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.820220 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.820340 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.820817 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:12:57.821361 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:12:57.821494 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.821582 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.821654 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:12:57.830636 1662 raft_consensus.cc:1261] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Refusing update from remote peer bb2b5cbcb93c4710a03b3dccf1cc8535: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:12:57.830636 1805 raft_consensus.cc:1261] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 1 FOLLOWER]: Refusing update from remote peer bb2b5cbcb93c4710a03b3dccf1cc8535: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:12:57.831880 1887 consensus_queue.cc:1035] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:12:57.832327 1883 consensus_queue.cc:1035] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:12:57.843704 1890 mvcc.cc:205] Tried to move back new op lower bound from 7315767205183516672 to 7315767205019058176. Current Snapshot: MvccSnapshot[applied={T|T < 7315767205183516672}]
I20260807 04:12:57.845031 1891 mvcc.cc:205] Tried to move back new op lower bound from 7315767205183516672 to 7315767205019058176. Current Snapshot: MvccSnapshot[applied={T|T < 7315767205183516672}]
I20260807 04:12:57.860754 1892 mvcc.cc:205] Tried to move back new op lower bound from 7315767205183516672 to 7315767205019058176. Current Snapshot: MvccSnapshot[applied={T|T < 7315767205183516672}]
I20260807 04:12:57.872622 1362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:12:57.872905 1362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:12:57.877151 1362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Sending DeleteTablet for 3 replicas of tablet e8168a2652794101a74017b6bf718ba2
I20260807 04:12:57.878270 1495 tablet_service.cc:1514] Processing DeleteTablet for tablet e8168a2652794101a74017b6bf718ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:12:57 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:54998
I20260807 04:12:57.878986 1784 tablet_service.cc:1514] Processing DeleteTablet for tablet e8168a2652794101a74017b6bf718ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:12:57 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:60728
I20260807 04:12:57.879340 1900 tablet_replica.cc:330] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: stopping tablet replica
I20260807 04:12:57.879798 1900 raft_consensus.cc:2229] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:12:57.879954 1901 tablet_replica.cc:330] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: stopping tablet replica
I20260807 04:12:57.880344 1901 raft_consensus.cc:2229] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:12:57.880595 1640 tablet_service.cc:1514] Processing DeleteTablet for tablet e8168a2652794101a74017b6bf718ba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:12:57 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:58704
I20260807 04:12:57.880823 1900 raft_consensus.cc:2258] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:12:57.880893 1901 raft_consensus.cc:2258] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:12:57.883190 1900 ts_tablet_manager.cc:1905] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:12:57.884824 1901 ts_tablet_manager.cc:1905] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:12:57.885782 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1427
I20260807 04:12:57.891898 1900 ts_tablet_manager.cc:1918] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:12:57.892110 1900 log.cc:1196] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/e8168a2652794101a74017b6bf718ba2
I20260807 04:12:57.892838 1900 ts_tablet_manager.cc:1939] T e8168a2652794101a74017b6bf718ba2 P bb2b5cbcb93c4710a03b3dccf1cc8535: Deleting consensus metadata
I20260807 04:12:57.893147 1901 ts_tablet_manager.cc:1918] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:12:57.893471 1901 log.cc:1196] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/e8168a2652794101a74017b6bf718ba2
I20260807 04:12:57.894112 1901 ts_tablet_manager.cc:1939] T e8168a2652794101a74017b6bf718ba2 P f48d92e63a154a61996e4776f3d024b9: Deleting consensus metadata
I20260807 04:12:57.895052 1346 catalog_manager.cc:4841] TS bb2b5cbcb93c4710a03b3dccf1cc8535 (127.18.193.1:38059): tablet e8168a2652794101a74017b6bf718ba2 (table test-table [id=3b9c121c189b4825807b0683ee33faa1]) successfully deleted
I20260807 04:12:57.896132 1347 catalog_manager.cc:4841] TS f48d92e63a154a61996e4776f3d024b9 (127.18.193.3:33851): tablet e8168a2652794101a74017b6bf718ba2 (table test-table [id=3b9c121c189b4825807b0683ee33faa1]) successfully deleted
I20260807 04:12:57.897418 1902 tablet_replica.cc:330] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: stopping tablet replica
I20260807 04:12:57.897910 1902 raft_consensus.cc:2229] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:12:57.898500 1902 raft_consensus.cc:2258] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:12:57.900518 1902 ts_tablet_manager.cc:1905] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 04:12:57 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:12:57.907851 1902 ts_tablet_manager.cc:1918] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:12:57.908035 1902 log.cc:1196] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/e8168a2652794101a74017b6bf718ba2
I20260807 04:12:57.908661 1902 ts_tablet_manager.cc:1939] T e8168a2652794101a74017b6bf718ba2 P 79f3613ddcb24cd6959496592242f197: Deleting consensus metadata
I20260807 04:12:57.910372 1345 catalog_manager.cc:4841] TS 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801): tablet e8168a2652794101a74017b6bf718ba2 (table test-table [id=3b9c121c189b4825807b0683ee33faa1]) successfully deleted
I20260807 04:12:57.917420 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:57.887072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.18.193.1:38059)
0807 04:12:57.887241 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:57.887250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:57.896652 (+ 9402us) server_negotiation.cc:407] Connection header received
0807 04:12:57.896918 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:57.896926 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:57.897056 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:57.897178 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:57.898937 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.900312 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.902405 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:57.903177 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:57.903355 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:57.909209 (+ 5854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:57.909228 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:57.909250 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:57.909314 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:57.914000 (+ 4686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:57.914884 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:57.914896 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:57.914907 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:57.915128 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:57.915520 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:57.915528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:57.915535 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:57.916075 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:57.916416 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:57.916692 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 04:12:57.916987 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:12:57.918251 1561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:58.020980 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1427
I20260807 04:12:58.042392 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1572
Aug 07 04:12:58 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:12:58.061851 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:58.043195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.18.193.2:36801)
0807 04:12:58.043376 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:58.043381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:58.043401 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:12:58.043484 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:58.043488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:58.043557 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:58.043643 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:58.045007 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:58.045825 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:58.047231 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:58.047608 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:58.047706 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:58.056266 (+ 8560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:58.056284 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:58.056301 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:58.056346 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:58.059383 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:58.060025 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:58.060031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:58.060034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:58.060152 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:58.060464 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:58.060468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:58.060470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:58.060939 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:58.061147 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:58.061418 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 04:12:58.061582 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:12:58.062578 1705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:58.153439 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1572
I20260807 04:12:58.172545 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1717
Aug 07 04:12:58 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:12:58.187608 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:58.173430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.18.193.3:33851)
0807 04:12:58.173656 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:58.173662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:58.173685 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:12:58.173762 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:58.173766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:58.173828 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:58.173922 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:58.175174 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:58.175888 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:58.177450 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:58.177857 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:58.177966 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:58.182428 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:58.182443 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:58.182460 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:58.182492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:58.185231 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:58.185892 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:58.185899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:58.185901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:58.186019 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:58.186327 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:58.186330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:58.186332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:58.186677 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:58.186890 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:58.187174 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 04:12:58.187328 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 04:12:58.188256 1850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:58.278087 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1717
I20260807 04:12:58.296753 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1325
I20260807 04:12:58.297905 1393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:12:58.412541 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1325
I20260807 04:12:58.436358 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38421
--webserver_interface=127.18.193.62
--webserver_port=43513
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:38421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:12:58.775857 1908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:58.776175 1908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:58.776329 1908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:58.783702 1908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:12:58.783810 1908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:58.783907 1908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:58.784014 1908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:12:58.784085 1908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:12:58.794127 1908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:38421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=43513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:58.795475 1908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:58.796849 1908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:58.822830 1916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:58.822922 1915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:58.822952 1918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:58.823859 1908 server_base.cc:1034] running on GCE node
I20260807 04:12:58.824906 1908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:58.826610 1908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:58.827821 1908 hybrid_clock.cc:648] HybridClock initialized: now 1786075978827781 us; error 36 us; skew 500 ppm
I20260807 04:12:58.828279 1908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:58 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075978, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:58.835814 1908 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:12:58.837841 1908 webserver.cc:460] Webserver started at http://127.18.193.62:43513/ using document root <none> and password file <none>
I20260807 04:12:58.838722 1908 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:58.838898 1908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:58.845429 1908 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:12:58.848692 1925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:58.850145 1908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:12:58.850334 1908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "75c2b73495e84fb08447fcab183b72b7"
format_stamp: "Formatted at 2026-08-07 04:12:55 on dist-test-slave-8n49"
server_key: "ee9e594e20e03e2ba2291ff3bb79e5f5"
server_key_iv: "e1d64b79f14fa5952e3aae3cbc33bb70"
server_key_version: "encryptionkey@0"
I20260807 04:12:58.851490 1908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:58.870083 1908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:58.874423 1908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:58.874694 1908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:58.894881 1908 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:38421
I20260807 04:12:58.894919 1980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:38421 every 8 connection(s)
I20260807 04:12:58.896524 1908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:12:58.903010 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1908
I20260807 04:12:58.903781 1981 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:12:58.903942 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:38059
--local_ip_for_outbound_sockets=127.18.193.1
--tserver_master_addrs=127.18.193.62:38421
--webserver_port=34231
--webserver_interface=127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:12:58.907374 1981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Bootstrap starting.
I20260807 04:12:58.928000 1981 log.cc:824] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:12:58.940231 1981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:12:58.940855 1981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Bootstrap complete.
I20260807 04:12:58.951117 1981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:58.952014 1981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c2b73495e84fb08447fcab183b72b7, State: Initialized, Role: FOLLOWER
I20260807 04:12:58.952714 1981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:58.952980 1981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:12:58.953132 1981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:12:58.953317 1981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:12:58.957710 1981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:58.958303 1981 leader_election.cc:302] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75c2b73495e84fb08447fcab183b72b7; no voters:
I20260807 04:12:58.958989 1981 leader_election.cc:288] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:12:58.959147 1985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:12:58.960516 1985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 LEADER]: Becoming Leader. State: Replica: 75c2b73495e84fb08447fcab183b72b7, State: Running, Role: LEADER
I20260807 04:12:58.961181 1985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:12:58.961611 1981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:12:58.963775 1987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c2b73495e84fb08447fcab183b72b7. Latest consensus state: current_term: 2 leader_uuid: "75c2b73495e84fb08447fcab183b72b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } } }
I20260807 04:12:58.964136 1987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:58.964766 1986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "75c2b73495e84fb08447fcab183b72b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } } }
I20260807 04:12:58.965174 1986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:12:58.973774 1994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:12:58.982451 1994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:12:58.984195 1994 catalog_manager.cc:1259] Loaded cluster ID: 26b2d60b0a5844aaae80c996c0acb9cf
I20260807 04:12:58.984354 1994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:12:58.987849 1994 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 04:12:59.006043 1994 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 04:12:59.015581 1994 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:12:59.017760 1994 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Loaded TSK: 0
I20260807 04:12:59.019304 1994 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:12:59.275095 1983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:59.275466 1983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:59.275660 1983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:59.283179 1983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:59.283340 1983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:59.283528 1983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:12:59.293809 1983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:38059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=34231
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:59.295125 1983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:59.296466 1983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:59.311271 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:59.311460 2013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:59.312422 2016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:59.312832 1983 server_base.cc:1034] running on GCE node
I20260807 04:12:59.313571 1983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:59.314713 1983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:59.315953 1983 hybrid_clock.cc:648] HybridClock initialized: now 1786075979315883 us; error 57 us; skew 500 ppm
I20260807 04:12:59.316306 1983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:59 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:59.320276 1983 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:12:59.322108 1983 webserver.cc:460] Webserver started at http://127.18.193.1:34231/ using document root <none> and password file <none>
I20260807 04:12:59.322968 1983 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:59.323117 1983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:59.330260 1983 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:12:59.333690 2023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:59.335371 1983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:12:59.335566 1983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "e8ebc115dced738463c268bc5fed9b9a"
server_key_iv: "d42645dc0440827bd9d5b532b4b98f28"
server_key_version: "encryptionkey@0"
I20260807 04:12:59.337013 1983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:59.368476 1983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:59.375173 1983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:59.375541 1983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:59.376719 1983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:59.378057 1983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:59.378153 1983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:59.378233 1983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:59.378288 1983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:59.413020 1983 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:38059
I20260807 04:12:59.413090 2145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:38059 every 8 connection(s)
I20260807 04:12:59.414495 1983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:12:59.422873 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 1983
I20260807 04:12:59.423792 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:36801
--local_ip_for_outbound_sockets=127.18.193.2
--tserver_master_addrs=127.18.193.62:38421
--webserver_port=38507
--webserver_interface=127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:12:59 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:59.440481 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:59.417520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:58067 (local address 127.18.193.62:38421)
0807 04:12:59.418076 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:59.418113 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:59.418753 (+ 640us) server_negotiation.cc:407] Connection header received
0807 04:12:59.420942 (+ 2189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:59.420956 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:59.421270 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:59.421635 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:59.423248 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.425382 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.426960 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.428080 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.428296 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:59.432157 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:59.432170 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:59.432193 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:59.432248 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:59.435575 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:59.436678 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:59.436686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:59.436692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:59.436866 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:59.437465 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:59.437469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:59.437471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:59.437998 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:59.438151 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:59.439140 (+ 989us) server_negotiation.cc:299] Negotiation successful
0807 04:12:59.439518 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":237,"threads_started":1}
I20260807 04:12:59.440994 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:59.417763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:58067)
0807 04:12:59.418569 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:59.418616 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:59.420359 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:59.421785 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:59.421798 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:59.422430 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:59.422958 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:59.422976 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.425572 (+ 2596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.425577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:59.426800 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:59.426813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.428306 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.428311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:59.428500 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:59.429258 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:59.429286 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:59.431841 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:59.435748 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:59.435760 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:59.435780 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:59.436451 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:59.437000 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:59.437008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:59.437015 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:59.437295 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:59.438169 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:59.438180 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:59.438634 (+ 454us) client_negotiation.cc:769] Sending connection context
0807 04:12:59.438960 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 04:12:59.439561 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":330,"threads_started":1}
I20260807 04:12:59.444000 2146 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38421
I20260807 04:12:59.444321 2146 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:59.445116 2146 heartbeater.cc:511] Master 127.18.193.62:38421 requested a full tablet report, sending...
I20260807 04:12:59.447532 1943 ts_manager.cc:194] Registered new tserver with Master: bb2b5cbcb93c4710a03b3dccf1cc8535 (127.18.193.1:38059)
I20260807 04:12:59.449103 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:58067
W20260807 04:12:59.767379 2152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:12:59.767648 2152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:12:59.767743 2152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:12:59.776067 2152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:12:59.776204 2152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:12:59.776321 2152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:12:59.786062 2152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:36801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=38507
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:12:59.787278 2152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:12:59.788539 2152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:12:59.802032 2157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:59.801990 2158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:12:59.802080 2160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:12:59.803468 2152 server_base.cc:1034] running on GCE node
I20260807 04:12:59.804095 2152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:12:59.805142 2152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:12:59.806370 2152 hybrid_clock.cc:648] HybridClock initialized: now 1786075979806284 us; error 70 us; skew 500 ppm
I20260807 04:12:59.806735 2152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:12:59 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:12:59.810101 2152 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:12:59.811635 2152 webserver.cc:460] Webserver started at http://127.18.193.2:38507/ using document root <none> and password file <none>
I20260807 04:12:59.812342 2152 fs_manager.cc:359] Metadata directory not provided
I20260807 04:12:59.812539 2152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:12:59.818079 2152 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 04:12:59.821022 2168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:59.822484 2152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:12:59.822675 2152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "79f3613ddcb24cd6959496592242f197"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "2f254b99a2e1d9ed0e2fe1c1a0200944"
server_key_iv: "d2bef970fffc5f5ce988dc170ca561ed"
server_key_version: "encryptionkey@0"
I20260807 04:12:59.823725 2152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:12:59.839427 2152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:12:59.844218 2152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:12:59.844521 2152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:12:59.845645 2152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:12:59.847131 2152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:12:59.847262 2152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:59.847410 2152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:12:59.847500 2152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:12:59.897710 2152 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:36801
I20260807 04:12:59.897755 2288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:36801 every 8 connection(s)
I20260807 04:12:59.899329 2152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:12:59.904373 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 2152
I20260807 04:12:59.905236 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:33851
--local_ip_for_outbound_sockets=127.18.193.3
--tserver_master_addrs=127.18.193.62:38421
--webserver_port=38149
--webserver_interface=127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:12:59 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:12:59.927763 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:59.902080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:41261 (local address 127.18.193.62:38421)
0807 04:12:59.902302 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 04:12:59.902308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:12:59.903603 (+ 1295us) server_negotiation.cc:407] Connection header received
0807 04:12:59.906969 (+ 3366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:12:59.906979 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:12:59.907104 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:12:59.907188 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:12:59.909693 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.910840 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.913469 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.914243 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.914357 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:59.919279 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:12:59.919286 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:12:59.919292 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:12:59.919338 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:12:59.923320 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:59.924495 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:59.924502 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:59.924507 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:59.924729 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:12:59.925322 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:12:59.925329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:12:59.925334 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:12:59.925741 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:12:59.925896 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 04:12:59.927007 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0807 04:12:59.927289 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:12:59.930469 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:12:59.902615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:41261)
0807 04:12:59.903431 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 04:12:59.903494 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 04:12:59.906593 (+ 3099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:12:59.907759 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:12:59.907774 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:12:59.908641 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:12:59.909477 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:59.909506 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.911088 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.911096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:59.913287 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:12:59.913305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:12:59.914717 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:12:59.914725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:12:59.915059 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:12:59.915920 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:12:59.915956 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:12:59.919011 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:12:59.923522 (+ 4511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:59.923534 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:59.923555 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:59.924342 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:59.924903 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:12:59.924910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:12:59.924916 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:12:59.925172 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:12:59.926000 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:12:59.926011 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:12:59.926482 (+ 471us) client_negotiation.cc:769] Sending connection context
0807 04:12:59.928263 (+ 1781us) client_negotiation.cc:240] Negotiation successful
0807 04:12:59.928938 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":351,"threads_started":1}
I20260807 04:12:59.932215 2289 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38421
I20260807 04:12:59.932615 2289 heartbeater.cc:464] Registering TS with master...
I20260807 04:12:59.933595 2289 heartbeater.cc:511] Master 127.18.193.62:38421 requested a full tablet report, sending...
I20260807 04:12:59.935477 1943 ts_manager.cc:194] Registered new tserver with Master: 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801)
I20260807 04:12:59.936856 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:41261
W20260807 04:13:00.259749 2292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:00.260016 2292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:00.260176 2292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:00.267516 2292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:00.267657 2292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:00.267848 2292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:13:00.277518 2292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:33851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=38149
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:00.278666 2292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:00.279867 2292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:00.292659 2300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:00.292713 2297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:00.292747 2298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:00.293529 2292 server_base.cc:1034] running on GCE node
I20260807 04:13:00.294113 2292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:00.295048 2292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:00.296275 2292 hybrid_clock.cc:648] HybridClock initialized: now 1786075980296232 us; error 68 us; skew 500 ppm
I20260807 04:13:00.296653 2292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:00 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075980, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:00.299937 2292 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:13:00.301486 2292 webserver.cc:460] Webserver started at http://127.18.193.3:38149/ using document root <none> and password file <none>
I20260807 04:13:00.302148 2292 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:00.302268 2292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:00.308037 2292 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:13:00.310989 2308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:00.312428 2292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:13:00.312644 2292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "f48d92e63a154a61996e4776f3d024b9"
format_stamp: "Formatted at 2026-08-07 04:12:57 on dist-test-slave-8n49"
server_key: "c8b06f4a22f5e61b80bc5a5a95c40f6d"
server_key_iv: "deba32761a499dda6cefde262bc36e81"
server_key_version: "encryptionkey@0"
I20260807 04:13:00.313633 2292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:00.340447 2292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:00.344834 2292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:00.345078 2292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:00.346081 2292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:00.347455 2292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:00.347553 2292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:00.347627 2292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:00.347674 2292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:00.381933 2292 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33851
I20260807 04:13:00.382025 2427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33851 every 8 connection(s)
I20260807 04:13:00.383396 2292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:13:00.388986 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 2292
Aug 07 04:13:00 dist-test-slave-8n49 krb5kdc[1311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075980, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:00.405416 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.390020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.18.193.62:38421)
0807 04:13:00.390783 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.390787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.390800 (+ 13us) server_negotiation.cc:407] Connection header received
0807 04:13:00.390841 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.390844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.390890 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.390979 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:00.392764 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.393495 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.395322 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.395734 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.395826 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.398786 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:00.398790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:00.398793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:00.398853 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:00.402398 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.403165 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.403169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.403170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.403274 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.403681 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.403686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.403689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.404015 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:00.404195 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.404662 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.404988 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":562,"threads_started":1}
I20260807 04:13:00.405824 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.385923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:37881 (local address 127.18.193.62:38421)
0807 04:13:00.386130 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.386137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.387289 (+ 1152us) server_negotiation.cc:407] Connection header received
0807 04:13:00.389094 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.389102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.389195 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.389316 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:00.391146 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.391847 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.393684 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.394109 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.394199 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.398152 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:00.398157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:00.398160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:00.398188 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:00.402013 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.402882 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.402886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.402888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.403003 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.403461 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.403464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.403466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.403796 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:00.403958 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.404801 (+ 843us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.405349 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":66}
I20260807 04:13:00.406427 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.386449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:37881)
0807 04:13:00.387128 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.387170 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.388831 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.389507 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.389515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.390092 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:13:00.390919 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.390938 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.392023 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.392030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.393536 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.393548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.394335 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.394344 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.394626 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.395317 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:13:00.395351 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:13:00.397949 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:13:00.402170 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:00.402177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:00.402190 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:00.402758 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:00.403114 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:00.403120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:00.403127 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:00.403337 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:00.403939 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:13:00.403945 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:13:00.404322 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.404643 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.405151 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":263,"threads_started":1}
I20260807 04:13:00.407833 2428 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38421
I20260807 04:13:00.408165 2428 heartbeater.cc:464] Registering TS with master...
I20260807 04:13:00.408890 2428 heartbeater.cc:511] Master 127.18.193.62:38421 requested a full tablet report, sending...
I20260807 04:13:00.410179 1943 ts_manager.cc:194] Registered new tserver with Master: f48d92e63a154a61996e4776f3d024b9 (127.18.193.3:33851)
I20260807 04:13:00.410990 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:37881
I20260807 04:13:00.420608 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:13:00.427127 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.421480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.18.193.62:38421)
0807 04:13:00.421720 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.421729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.422072 (+ 343us) server_negotiation.cc:407] Connection header received
0807 04:13:00.422253 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.422257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.422394 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.422522 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:13:00.422921 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.423666 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.425109 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.425526 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.425620 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.426063 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:13:00.426518 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:13:00.426628 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.426706 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.426767 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:13:00.428018 1943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:13:00.435477 1943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:13:00.452231 2146 heartbeater.cc:503] Master 127.18.193.62:38421 was elected leader, sending a full tablet report...
I20260807 04:13:00.468714 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.447763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33851 (local address 127.0.0.1:60736)
0807 04:13:00.458698 (+ 10935us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.458757 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.458970 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.460930 (+ 1960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.460937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.460970 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:00.461571 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.461587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.463773 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.463778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.466291 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.466301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.467630 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.467636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.467826 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.467889 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.468033 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.468271 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10586,"mutex_wait_us":110,"thread_start_us":11499,"threads_started":1}
I20260807 04:13:00.469326 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.450543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.18.193.2:36801)
0807 04:13:00.459843 (+ 9300us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.459853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.460605 (+ 752us) server_negotiation.cc:407] Connection header received
0807 04:13:00.460716 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.460723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.460940 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.461105 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:00.462211 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.463707 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.466153 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.467645 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.467782 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.468734 (+ 952us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.468813 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.469001 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9122,"thread_start_us":8985,"threads_started":1}
I20260807 04:13:00.469342 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.447979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.18.193.1:38059)
0807 04:13:00.457775 (+ 9796us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.457787 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.459733 (+ 1946us) server_negotiation.cc:407] Connection header received
0807 04:13:00.459851 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.459861 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.460192 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.460391 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:00.462029 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.463368 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.466524 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.467960 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.468099 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.468307 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.468907 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.469084 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9563,"thread_start_us":9814,"threads_started":1}
I20260807 04:13:00.469784 2440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.447860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.18.193.3:33851)
0807 04:13:00.460289 (+ 12429us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.460297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.460331 (+ 34us) server_negotiation.cc:407] Connection header received
0807 04:13:00.460426 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.460443 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.460688 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.460888 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:00.461860 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.463603 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.466478 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.467413 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.467569 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.467814 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.469136 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.469302 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12254,"thread_start_us":12098,"threads_started":1}
I20260807 04:13:00.469062 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.449860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:36801 (local address 127.0.0.1:58726)
0807 04:13:00.460235 (+ 10375us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.460257 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.460399 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.461404 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.461412 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.461444 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:00.461992 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.462008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.463813 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.463824 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.465951 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.465965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.468099 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.468106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.468299 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.468338 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.468430 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.468588 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10253,"mutex_wait_us":5,"thread_start_us":10287,"threads_started":1}
I20260807 04:13:00.470337 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.447804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55022)
0807 04:13:00.459581 (+ 11777us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.459598 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.459715 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.461180 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.461184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.461205 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:00.461797 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.461813 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.463536 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.463539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.466286 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.466300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.468481 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.468486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.468680 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.468728 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.469760 (+ 1032us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.469920 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11645,"mutex_wait_us":50,"thread_start_us":10402,"threads_started":1}
I20260807 04:13:00.473877 2075 tablet_service.cc:1467] Processing CreateTablet for tablet 634fcadc457d40689db005b5920b8455 (DEFAULT_TABLE table=test-table [id=b76dae81aaf845dd89776a8d04bdea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:00.474128 2358 tablet_service.cc:1467] Processing CreateTablet for tablet 634fcadc457d40689db005b5920b8455 (DEFAULT_TABLE table=test-table [id=b76dae81aaf845dd89776a8d04bdea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:00.475255 2075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634fcadc457d40689db005b5920b8455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:00.474964 2218 tablet_service.cc:1467] Processing CreateTablet for tablet 634fcadc457d40689db005b5920b8455 (DEFAULT_TABLE table=test-table [id=b76dae81aaf845dd89776a8d04bdea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:00.475503 2358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634fcadc457d40689db005b5920b8455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:00.476187 2218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634fcadc457d40689db005b5920b8455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:00.488258 2447 tablet_bootstrap.cc:492] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Bootstrap starting.
I20260807 04:13:00.491048 2448 tablet_bootstrap.cc:492] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Bootstrap starting.
I20260807 04:13:00.493243 2449 tablet_bootstrap.cc:492] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Bootstrap starting.
I20260807 04:13:00.493414 2447 tablet_bootstrap.cc:654] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:00.494556 2448 tablet_bootstrap.cc:654] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:00.494797 2447 log.cc:824] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:00.495744 2448 log.cc:824] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:00.496380 2449 tablet_bootstrap.cc:654] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:00.497686 2449 log.cc:824] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:00.498775 2447 tablet_bootstrap.cc:492] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: No bootstrap required, opened a new log
I20260807 04:13:00.499056 2447 ts_tablet_manager.cc:1397] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 04:13:00.499366 2448 tablet_bootstrap.cc:492] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: No bootstrap required, opened a new log
I20260807 04:13:00.499687 2448 ts_tablet_manager.cc:1397] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 04:13:00.500407 2449 tablet_bootstrap.cc:492] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: No bootstrap required, opened a new log
I20260807 04:13:00.500711 2449 ts_tablet_manager.cc:1397] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 04:13:00.507709 2449 raft_consensus.cc:348] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.508381 2449 raft_consensus.cc:374] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:00.508574 2449 raft_consensus.cc:729] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79f3613ddcb24cd6959496592242f197, State: Initialized, Role: FOLLOWER
I20260807 04:13:00.509349 2449 consensus_queue.cc:260] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.510612 2289 heartbeater.cc:503] Master 127.18.193.62:38421 was elected leader, sending a full tablet report...
I20260807 04:13:00.510372 2448 raft_consensus.cc:348] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.510810 2448 raft_consensus.cc:374] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:00.510918 2448 raft_consensus.cc:729] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f48d92e63a154a61996e4776f3d024b9, State: Initialized, Role: FOLLOWER
I20260807 04:13:00.510372 2447 raft_consensus.cc:348] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.511339 2449 ts_tablet_manager.cc:1428] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260807 04:13:00.511475 2447 raft_consensus.cc:374] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:00.511612 2447 raft_consensus.cc:729] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2b5cbcb93c4710a03b3dccf1cc8535, State: Initialized, Role: FOLLOWER
I20260807 04:13:00.511739 2448 consensus_queue.cc:260] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.512218 2447 consensus_queue.cc:260] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.513006 2428 heartbeater.cc:503] Master 127.18.193.62:38421 was elected leader, sending a full tablet report...
I20260807 04:13:00.513664 2448 ts_tablet_manager.cc:1428] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260807 04:13:00.515467 2447 ts_tablet_manager.cc:1428] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260807 04:13:00.562093 2454 raft_consensus.cc:482] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:13:00.562412 2454 raft_consensus.cc:504] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.563773 2455 raft_consensus.cc:482] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:13:00.564051 2455 raft_consensus.cc:504] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.564296 2454 leader_election.cc:288] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb2b5cbcb93c4710a03b3dccf1cc8535 (127.18.193.1:38059), 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801)
I20260807 04:13:00.566097 2455 leader_election.cc:288] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f48d92e63a154a61996e4776f3d024b9 (127.18.193.3:33851), 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801)
I20260807 04:13:00.574553 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.564444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.18.193.3:52427)
0807 04:13:00.564846 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.564867 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.564998 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.565689 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.565696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.565722 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:00.565951 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.565972 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.568224 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.568233 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.570952 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.570973 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.573496 (+ 2523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.573505 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.573775 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.573828 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.573954 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.574156 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 04:13:00.575493 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.564978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:52427 (local address 127.18.193.1:38059)
0807 04:13:00.565226 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.565234 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.565256 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:13:00.565331 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.565336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.565412 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.565504 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:00.566898 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.568041 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.571202 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.573078 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.573276 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.573389 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.574951 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.575146 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:13:00.579408 2097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634fcadc457d40689db005b5920b8455" candidate_uuid: "f48d92e63a154a61996e4776f3d024b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" is_pre_election: true
I20260807 04:13:00.580055 2097 raft_consensus.cc:2454] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f48d92e63a154a61996e4776f3d024b9 in term 0.
I20260807 04:13:00.581024 2311 leader_election.cc:302] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb2b5cbcb93c4710a03b3dccf1cc8535, f48d92e63a154a61996e4776f3d024b9; no voters:
I20260807 04:13:00.581516 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.565450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:59349 (local address 127.18.193.2:36801)
0807 04:13:00.565669 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.565675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.565723 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:13:00.565800 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.565805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.565881 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.565977 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:00.566802 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.567933 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.578639 (+ 10706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.580333 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.580463 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.580569 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.581129 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.581264 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:13:00.581514 2454 raft_consensus.cc:2781] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:13:00.581673 2454 raft_consensus.cc:482] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:13:00.581785 2454 raft_consensus.cc:3037] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:00.583357 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.564785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:36801 (local address 127.18.193.3:59349)
0807 04:13:00.565540 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.565560 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.565660 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.566157 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.566162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.566191 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:00.566579 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.566592 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.568140 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.568145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.578385 (+ 10240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.578407 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.580697 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.580704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.580912 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.580945 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.581730 (+ 785us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.581902 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":7496,"thread_start_us":930,"threads_started":1}
I20260807 04:13:00.584595 2240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634fcadc457d40689db005b5920b8455" candidate_uuid: "f48d92e63a154a61996e4776f3d024b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f3613ddcb24cd6959496592242f197" is_pre_election: true
I20260807 04:13:00.585178 2240 raft_consensus.cc:2454] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f48d92e63a154a61996e4776f3d024b9 in term 0.
I20260807 04:13:00.586227 2454 raft_consensus.cc:504] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.587164 2454 leader_election.cc:288] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [CANDIDATE]: Term 1 election: Requested vote from peers bb2b5cbcb93c4710a03b3dccf1cc8535 (127.18.193.1:38059), 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801)
I20260807 04:13:00.587459 2097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634fcadc457d40689db005b5920b8455" candidate_uuid: "f48d92e63a154a61996e4776f3d024b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535"
I20260807 04:13:00.587678 2097 raft_consensus.cc:3037] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:00.587700 2240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634fcadc457d40689db005b5920b8455" candidate_uuid: "f48d92e63a154a61996e4776f3d024b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f3613ddcb24cd6959496592242f197"
I20260807 04:13:00.587913 2240 raft_consensus.cc:3037] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:00.588897 2440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.566725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:55179 (local address 127.18.193.3:33851)
0807 04:13:00.566906 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.566914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.578337 (+ 11423us) server_negotiation.cc:407] Connection header received
0807 04:13:00.579244 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.579249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.579326 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.579389 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:00.581559 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.583430 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.586418 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.587643 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.587804 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.587886 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.588504 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.588669 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 04:13:00.589432 2459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.566581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:33827 (local address 127.18.193.2:36801)
0807 04:13:00.580382 (+ 13801us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.580389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.581834 (+ 1445us) server_negotiation.cc:407] Connection header received
0807 04:13:00.581892 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.581896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.581983 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.582079 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:00.582886 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.584064 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.586508 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.587810 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.588009 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.588085 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.588951 (+ 866us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.589093 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13671,"thread_start_us":13798,"threads_started":1}
I20260807 04:13:00.591281 2240 raft_consensus.cc:2454] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f48d92e63a154a61996e4776f3d024b9 in term 1.
I20260807 04:13:00.591725 2097 raft_consensus.cc:2454] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f48d92e63a154a61996e4776f3d024b9 in term 1.
I20260807 04:13:00.591727 2310 leader_election.cc:302] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79f3613ddcb24cd6959496592242f197, f48d92e63a154a61996e4776f3d024b9; no voters:
I20260807 04:13:00.592060 2454 raft_consensus.cc:2781] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:13:00.591832 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.566270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33851 (local address 127.18.193.1:55179)
0807 04:13:00.578214 (+ 11944us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.578239 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.579073 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.579613 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.579618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.579642 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:00.581363 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.581376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.583855 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.583861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.586232 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.586249 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.588054 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.588062 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.588276 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.588323 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.589102 (+ 779us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.589218 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11748,"thread_start_us":12081,"threads_started":1}
I20260807 04:13:00.591871 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.566479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:36801 (local address 127.18.193.1:33827)
0807 04:13:00.581089 (+ 14610us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.581109 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:00.581720 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:00.582345 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:00.582348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:00.582367 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:00.582674 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.582685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.584246 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.584251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.586300 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:00.586334 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.588280 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.588287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:00.588495 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.588527 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:13:00.588651 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 04:13:00.588795 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14452,"thread_start_us":14250,"threads_started":1}
I20260807 04:13:00.592657 2454 raft_consensus.cc:686] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 1 LEADER]: Becoming Leader. State: Replica: f48d92e63a154a61996e4776f3d024b9, State: Running, Role: LEADER
I20260807 04:13:00.593293 2240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634fcadc457d40689db005b5920b8455" candidate_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f3613ddcb24cd6959496592242f197" is_pre_election: true
I20260807 04:13:00.593273 2454 consensus_queue.cc:237] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } }
I20260807 04:13:00.593608 2240 raft_consensus.cc:2379] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb2b5cbcb93c4710a03b3dccf1cc8535 in current term 1: Already voted for candidate f48d92e63a154a61996e4776f3d024b9 in this term.
I20260807 04:13:00.593485 2379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634fcadc457d40689db005b5920b8455" candidate_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f48d92e63a154a61996e4776f3d024b9" is_pre_election: true
I20260807 04:13:00.594588 2028 leader_election.cc:302] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb2b5cbcb93c4710a03b3dccf1cc8535; no voters: 79f3613ddcb24cd6959496592242f197, f48d92e63a154a61996e4776f3d024b9
I20260807 04:13:00.595010 2455 raft_consensus.cc:2726] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 04:13:00.598690 1941 catalog_manager.cc:5495] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f48d92e63a154a61996e4776f3d024b9 (127.18.193.3). New cstate: current_term: 1 leader_uuid: "f48d92e63a154a61996e4776f3d024b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f48d92e63a154a61996e4776f3d024b9" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 } health_report { overall_health: UNKNOWN } } }
I20260807 04:13:00.625766 2440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.619959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.18.193.3:33851)
0807 04:13:00.620172 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.620176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.620277 (+ 101us) server_negotiation.cc:407] Connection header received
0807 04:13:00.620515 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.620519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.620683 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.620816 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:13:00.621226 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.622017 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.623475 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.623858 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.623951 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.624524 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:13:00.625120 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:13:00.625228 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.625350 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.625414 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 04:13:00.633906 2240 raft_consensus.cc:1261] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Refusing update from remote peer f48d92e63a154a61996e4776f3d024b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:13:00.633965 2097 raft_consensus.cc:1261] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Refusing update from remote peer f48d92e63a154a61996e4776f3d024b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 04:13:00.635038 2429 tablet.cc:2369] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:13:00.634990 2461 consensus_queue.cc:1035] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:13:00.635823 2454 consensus_queue.cc:1035] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79f3613ddcb24cd6959496592242f197" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:13:00.650560 2465 mvcc.cc:205] Tried to move back new op lower bound from 7315767216666439680 to 7315767216515379200. Current Snapshot: MvccSnapshot[applied={T|T < 7315767216666439680}]
I20260807 04:13:00.653107 2464 mvcc.cc:205] Tried to move back new op lower bound from 7315767216666439680 to 7315767216515379200. Current Snapshot: MvccSnapshot[applied={T|T < 7315767216666439680}]
I20260807 04:13:00.670367 1941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:13:00.670687 1941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:13:00.674594 1941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Sending DeleteTablet for 3 replicas of tablet 634fcadc457d40689db005b5920b8455
I20260807 04:13:00.675698 2358 tablet_service.cc:1514] Processing DeleteTablet for tablet 634fcadc457d40689db005b5920b8455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:00 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:60736
I20260807 04:13:00.676152 2218 tablet_service.cc:1514] Processing DeleteTablet for tablet 634fcadc457d40689db005b5920b8455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:00 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:58726
I20260807 04:13:00.676460 2075 tablet_service.cc:1514] Processing DeleteTablet for tablet 634fcadc457d40689db005b5920b8455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:00 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:55022
I20260807 04:13:00.676707 2474 tablet_replica.cc:330] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: stopping tablet replica
I20260807 04:13:00.677126 2474 raft_consensus.cc:2229] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:13:00.677496 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1983
I20260807 04:13:00.677860 2474 raft_consensus.cc:2258] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:00.680073 2474 ts_tablet_manager.cc:1905] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:00.687768 2474 ts_tablet_manager.cc:1918] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:13:00.688004 2474 log.cc:1196] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/634fcadc457d40689db005b5920b8455
I20260807 04:13:00.688694 2474 ts_tablet_manager.cc:1939] T 634fcadc457d40689db005b5920b8455 P f48d92e63a154a61996e4776f3d024b9: Deleting consensus metadata
I20260807 04:13:00.690723 1929 catalog_manager.cc:4841] TS f48d92e63a154a61996e4776f3d024b9 (127.18.193.3:33851): tablet 634fcadc457d40689db005b5920b8455 (table test-table [id=b76dae81aaf845dd89776a8d04bdea5a]) successfully deleted
I20260807 04:13:00.700717 2476 tablet_replica.cc:330] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: stopping tablet replica
I20260807 04:13:00.701226 2476 raft_consensus.cc:2229] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:13:00.701809 2476 raft_consensus.cc:2258] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:00.703015 2475 tablet_replica.cc:330] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: stopping tablet replica
I20260807 04:13:00.703471 2475 raft_consensus.cc:2229] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:13:00.704005 2475 raft_consensus.cc:2258] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:00.704339 2476 ts_tablet_manager.cc:1905] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:00.706210 2475 ts_tablet_manager.cc:1905] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:00.707242 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.679431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55028 (local address 127.18.193.1:38059)
0807 04:13:00.679657 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.679666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.679690 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:13:00.679763 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.679771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.679869 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.680005 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:00.682656 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.684276 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.687161 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.688859 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.689041 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.696380 (+ 7339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:00.696408 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:00.696432 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:00.696501 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:00.702149 (+ 5648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.703879 (+ 1730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.703888 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.703891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.704071 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.704705 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.704711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.704714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.705224 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:00.705537 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.706541 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.706824 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:13:00.708693 2140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:00.714795 2475 ts_tablet_manager.cc:1918] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:13:00.714978 2475 log.cc:1196] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/634fcadc457d40689db005b5920b8455
I20260807 04:13:00.715615 2475 ts_tablet_manager.cc:1939] T 634fcadc457d40689db005b5920b8455 P 79f3613ddcb24cd6959496592242f197: Deleting consensus metadata
I20260807 04:13:00.717684 1927 catalog_manager.cc:4841] TS 79f3613ddcb24cd6959496592242f197 (127.18.193.2:36801): tablet 634fcadc457d40689db005b5920b8455 (table test-table [id=b76dae81aaf845dd89776a8d04bdea5a]) successfully deleted
I20260807 04:13:00.805599 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1983
I20260807 04:13:00.826303 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 2152
W20260807 04:13:00.826422 1928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.193.1:38059 (error 108)
I20260807 04:13:00.829002 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.828462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55032)
0807 04:13:00.828680 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.828811 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 04:13:00.829270 1928 catalog_manager.cc:4584] TS bb2b5cbcb93c4710a03b3dccf1cc8535 (127.18.193.1:38059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 634fcadc457d40689db005b5920b8455: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
I20260807 04:13:00.841652 2459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.827340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.18.193.2:36801)
0807 04:13:00.827997 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.828005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.828027 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:13:00.828096 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.828101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.828199 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.828315 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:00.830204 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.830921 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.832247 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.832901 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.833045 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.835777 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:00.835791 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:00.835804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:00.835841 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:00.838664 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.839377 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.839384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.839386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.839531 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.839909 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.839915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.839918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.840283 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:00.840629 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.841069 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.841304 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505}
I20260807 04:13:00.842374 2284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:00.883248 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.882638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55036)
0807 04:13:00.882904 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.883024 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 04:13:00.884452 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.884060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55050)
0807 04:13:00.884227 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.884324 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 04:13:00.945550 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 2152
I20260807 04:13:00.945956 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.945374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55058)
0807 04:13:00.945631 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.945750 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260807 04:13:00.947736 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.947325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55070)
0807 04:13:00.947486 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:00.947585 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 04:13:00.969204 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 2292
I20260807 04:13:00.985673 2440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:00.970111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.18.193.3:33851)
0807 04:13:00.970355 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:00.970361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:00.970385 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:13:00.970543 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:00.970551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:00.970645 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:00.970766 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:00.972274 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.973444 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.974755 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:00.975482 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:00.975643 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:00.978292 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:00.978310 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:00.978331 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:00.978381 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:00.982324 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.983053 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.983063 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.983067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.983223 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:00.983657 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:00.983663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:00.983668 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:00.984222 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:00.984610 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:00.985065 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 04:13:00.985291 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 04:13:00.986470 2423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:01.028821 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:01.028283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55076)
0807 04:13:01.028509 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:01.028662 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 04:13:01.029958 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:01.029567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55082)
0807 04:13:01.029773 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:01.029844 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 04:13:01.113194 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 2292
I20260807 04:13:01.137220 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 1908
I20260807 04:13:01.138476 1975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:01.252084 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:01.251402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:38059 (local address 127.0.0.1:55094)
0807 04:13:01.251620 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:01.251722 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:38059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 04:13:01.252199 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 1908
I20260807 04:13:01.277179 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38421
--webserver_interface=127.18.193.62
--webserver_port=43513
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:38421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:13:01.618178 2483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:01.618510 2483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:01.618604 2483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:01.625862 2483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:13:01.625974 2483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:01.626021 2483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:01.626081 2483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:13:01.626129 2483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:13:01.636191 2483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:38421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=43513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:01.637619 2483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:01.638878 2483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:01.651113 2490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:01.651386 2492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:01.651252 2488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:01.652072 2483 server_base.cc:1034] running on GCE node
I20260807 04:13:01.652959 2483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:01.654521 2483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:01.655725 2483 hybrid_clock.cc:648] HybridClock initialized: now 1786075981655677 us; error 39 us; skew 500 ppm
I20260807 04:13:01.656090 2483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:01 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:01.659310 2483 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:13:01.660858 2483 webserver.cc:460] Webserver started at http://127.18.193.62:43513/ using document root <none> and password file <none>
I20260807 04:13:01.661567 2483 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:01.661720 2483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:01.667222 2483 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 04:13:01.670157 2499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:01.671838 2483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:13:01.672050 2483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "75c2b73495e84fb08447fcab183b72b7"
format_stamp: "Formatted at 2026-08-07 04:12:55 on dist-test-slave-8n49"
server_key: "ee9e594e20e03e2ba2291ff3bb79e5f5"
server_key_iv: "e1d64b79f14fa5952e3aae3cbc33bb70"
server_key_version: "encryptionkey@0"
I20260807 04:13:01.673211 2483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:01.688019 2483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:01.692029 2483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:01.692312 2483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:01.711912 2483 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:38421
I20260807 04:13:01.711920 2555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:38421 every 8 connection(s)
I20260807 04:13:01.713546 2483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:13:01.720320 2556 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:13:01.723755 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 2483
I20260807 04:13:01.723807 2556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Bootstrap starting.
I20260807 04:13:01.724880 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:38059
--local_ip_for_outbound_sockets=127.18.193.1
--tserver_master_addrs=127.18.193.62:38421
--webserver_port=34231
--webserver_interface=127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:13:01.749065 2556 log.cc:824] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:01.765669 2556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:13:01.766263 2556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: Bootstrap complete.
I20260807 04:13:01.775899 2556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:13:01.777026 2556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c2b73495e84fb08447fcab183b72b7, State: Initialized, Role: FOLLOWER
I20260807 04:13:01.777694 2556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:13:01.777941 2556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:13:01.778074 2556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:13:01.778281 2556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 2 FOLLOWER]: Advancing to term 3
I20260807 04:13:01.783890 2556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:13:01.784469 2556 leader_election.cc:302] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75c2b73495e84fb08447fcab183b72b7; no voters:
I20260807 04:13:01.785254 2556 leader_election.cc:288] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 04:13:01.785398 2560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 04:13:01.787776 2556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:13:01.792606 2560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [term 3 LEADER]: Becoming Leader. State: Replica: 75c2b73495e84fb08447fcab183b72b7, State: Running, Role: LEADER
I20260807 04:13:01.793479 2560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } }
I20260807 04:13:01.797876 2561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "75c2b73495e84fb08447fcab183b72b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } } }
I20260807 04:13:01.798246 2561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:13:01.798919 2562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c2b73495e84fb08447fcab183b72b7. Latest consensus state: current_term: 3 leader_uuid: "75c2b73495e84fb08447fcab183b72b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c2b73495e84fb08447fcab183b72b7" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38421 } } }
I20260807 04:13:01.799214 2562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:13:01.814560 2574 catalog_manager.cc:1259] Loaded cluster ID: 26b2d60b0a5844aaae80c996c0acb9cf
I20260807 04:13:01.814685 2574 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: loading cluster ID for follower catalog manager: success
W20260807 04:13:01.816910 2574 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 75c2b73495e84fb08447fcab183b72b7: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260807 04:13:01.817101 2574 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260807 04:13:01.817332 2575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:13:01.823011 2575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:13:01.824283 2575 catalog_manager.cc:1259] Loaded cluster ID: 26b2d60b0a5844aaae80c996c0acb9cf
I20260807 04:13:01.824386 2575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 04:13:01.826118 2575 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f51320eb9ad google::LogMessage::Fail() at ??:0
@ 0x7f51320ee88a google::LogMessage::SendToLog() at ??:0
@ 0x7f51320eb4de google::LogMessage::Flush() at ??:0
@ 0x7f51320eca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5147b22512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5147b1f976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5147ba7610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5147ba7439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5134d37345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5134d3a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5134d3a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5134d15eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5137fef6db start_thread at ??:0
@ 0x7f5130f8561f clone at ??:0
W20260807 04:13:02.145226 2558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:02.145643 2558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:02.145820 2558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:02.153409 2558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:02.153580 2558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:02.153782 2558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:13:02.163726 2558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:38059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=34231
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:02.165143 2558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:02.166507 2558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:02.179973 2603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:02.180001 2601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:02.180050 2600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:02.180750 2558 server_base.cc:1034] running on GCE node
I20260807 04:13:02.181572 2558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:02.182531 2558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:02.183794 2558 hybrid_clock.cc:648] HybridClock initialized: now 1786075982183738 us; error 57 us; skew 500 ppm
I20260807 04:13:02.184149 2558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:02 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:02.187502 2558 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:13:02.189079 2558 webserver.cc:460] Webserver started at http://127.18.193.1:34231/ using document root <none> and password file <none>
I20260807 04:13:02.189821 2558 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:02.189980 2558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:02.195489 2558 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:13:02.198451 2610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:02.199741 2558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:13:02.199916 2558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "bb2b5cbcb93c4710a03b3dccf1cc8535"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "e8ebc115dced738463c268bc5fed9b9a"
server_key_iv: "d42645dc0440827bd9d5b532b4b98f28"
server_key_version: "encryptionkey@0"
I20260807 04:13:02.200992 2558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:02.231200 2558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:02.235569 2558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:02.235893 2558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:02.236972 2558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:02.238927 2617 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 04:13:02.244494 2617 ts_tablet_manager.cc:1852] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 04:13:02.245621 2617 ts_tablet_manager.cc:1905] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:02.253376 2617 ts_tablet_manager.cc:1918] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:13:02.253571 2617 log.cc:1196] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/634fcadc457d40689db005b5920b8455
I20260807 04:13:02.253999 2617 ts_tablet_manager.cc:1939] T 634fcadc457d40689db005b5920b8455 P bb2b5cbcb93c4710a03b3dccf1cc8535: Deleting consensus metadata
I20260807 04:13:02.254388 2558 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 04:13:02.254552 2558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260807 04:13:02.254724 2558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:02.254817 2558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:02.289950 2558 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:38059
I20260807 04:13:02.290009 2730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:38059 every 8 connection(s)
I20260807 04:13:02.291337 2558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:13:02.297020 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:02.294688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:53801)
0807 04:13:02.295467 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:02.295717 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":298,"threads_started":1}
W20260807 04:13:02.297775 2731 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:38421 (0 consecutive failures): Network error: Failed to ping master at 127.18.193.62:38421: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
I20260807 04:13:02.298473 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 2558
I20260807 04:13:02.298672 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:02.298307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:56487)
0807 04:13:02.298462 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:02.298539 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 04:13:02.299149 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:36801
--local_ip_for_outbound_sockets=127.18.193.2
--tserver_master_addrs=127.18.193.62:38421
--webserver_port=38507
--webserver_interface=127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:13:02.299537 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:02.299192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:35495)
0807 04:13:02.299331 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:02.299424 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:13:02.299779 2731 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:13:02.638139 2734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:02.638443 2734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:02.638545 2734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:02.645742 2734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:02.645951 2734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:02.646085 2734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:13:02.656023 2734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:36801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=38507
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:02.657404 2734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:02.658674 2734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:02.672097 2740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:02.672150 2744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:02.672186 2742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:02.673123 2734 server_base.cc:1034] running on GCE node
I20260807 04:13:02.673834 2734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:02.674870 2734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:02.676095 2734 hybrid_clock.cc:648] HybridClock initialized: now 1786075982676042 us; error 42 us; skew 500 ppm
I20260807 04:13:02.676440 2734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:02 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:02.679828 2734 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:13:02.681399 2734 webserver.cc:460] Webserver started at http://127.18.193.2:38507/ using document root <none> and password file <none>
I20260807 04:13:02.682127 2734 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:02.682276 2734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:02.687889 2734 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 04:13:02.690755 2751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:02.692166 2734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:13:02.692358 2734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "79f3613ddcb24cd6959496592242f197"
format_stamp: "Formatted at 2026-08-07 04:12:56 on dist-test-slave-8n49"
server_key: "2f254b99a2e1d9ed0e2fe1c1a0200944"
server_key_iv: "d2bef970fffc5f5ce988dc170ca561ed"
server_key_version: "encryptionkey@0"
I20260807 04:13:02.693514 2734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:02.708493 2734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:02.712972 2734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:02.713284 2734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:02.714347 2734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:02.715823 2734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:02.715955 2734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:02.716091 2734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:02.716192 2734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:02.754019 2734 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:36801
I20260807 04:13:02.754076 2875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:36801 every 8 connection(s)
I20260807 04:13:02.755594 2734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:13:02.756143 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 2734
I20260807 04:13:02.756974 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:33851
--local_ip_for_outbound_sockets=127.18.193.3
--tserver_master_addrs=127.18.193.62:38421
--webserver_port=38149
--webserver_interface=127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:13:02.761296 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:02.759176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:46751)
0807 04:13:02.759989 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:02.760229 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":325,"threads_started":1}
W20260807 04:13:02.762040 2876 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:38421 (0 consecutive failures): Network error: Failed to ping master at 127.18.193.62:38421: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
I20260807 04:13:02.762975 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:02.762619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:57563)
0807 04:13:02.762788 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:02.762859 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 04:13:02.763881 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:02.763545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:37091)
0807 04:13:02.763688 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:02.763773 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:13:02.764132 2876 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:13:03.098606 2879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:03.098881 2879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:03.098975 2879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:03.106310 2879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:03.106431 2879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:03.106571 2879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:13:03.116478 2879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:33851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=38149
--tserver_master_addrs=127.18.193.62:38421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:03.117871 2879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:03.119166 2879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:03.132478 2884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:03.132530 2887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:03.132440 2885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:03.133462 2879 server_base.cc:1034] running on GCE node
I20260807 04:13:03.134120 2879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:03.135116 2879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:03.136332 2879 hybrid_clock.cc:648] HybridClock initialized: now 1786075983136274 us; error 41 us; skew 500 ppm
I20260807 04:13:03.136736 2879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:03 dist-test-slave-8n49 krb5kdc[1311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786075983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:03.140136 2879 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:13:03.141827 2879 webserver.cc:460] Webserver started at http://127.18.193.3:38149/ using document root <none> and password file <none>
I20260807 04:13:03.142572 2879 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:03.142717 2879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:03.148360 2879 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:13:03.151077 2896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:03.152468 2879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:13:03.152700 2879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "f48d92e63a154a61996e4776f3d024b9"
format_stamp: "Formatted at 2026-08-07 04:12:57 on dist-test-slave-8n49"
server_key: "c8b06f4a22f5e61b80bc5a5a95c40f6d"
server_key_iv: "deba32761a499dda6cefde262bc36e81"
server_key_version: "encryptionkey@0"
I20260807 04:13:03.153725 2879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:03.175613 2879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:03.180337 2879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:03.180773 2879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:03.181895 2879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:03.183344 2879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:03.183442 2879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:03.183586 2879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:03.183681 2879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:03.219112 2879 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33851
I20260807 04:13:03.219206 3015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33851 every 8 connection(s)
I20260807 04:13:03.220685 2879 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:13:03.225659 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:03.223648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:40839)
0807 04:13:03.224479 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:03.224793 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":353,"threads_started":1}
I20260807 04:13:03.225953 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 2879
W20260807 04:13:03.226419 3016 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:38421 (0 consecutive failures): Network error: Failed to ping master at 127.18.193.62:38421: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
I20260807 04:13:03.227322 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:03.226978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:39649)
0807 04:13:03.227136 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:03.227210 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 04:13:03.228222 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:03.227900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:44999)
0807 04:13:03.228037 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:03.228122 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:13:03.228480 3016 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 04:13:03.301262 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:03.300354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:52821)
0807 04:13:03.300878 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:03.301001 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":288,"threads_started":1}
I20260807 04:13:03.765688 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:03.764796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:52547)
0807 04:13:03.765291 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:03.765446 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":239,"threads_started":1}
I20260807 04:13:04.230036 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:04.229189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:43783)
0807 04:13:04.229697 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:04.229797 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":265,"threads_started":1}
I20260807 04:13:04.303155 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:04.302359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:54819)
0807 04:13:04.302900 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:04.302995 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":280,"threads_started":1}
I20260807 04:13:04.767431 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:04.766660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:39681)
0807 04:13:04.767159 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:04.767248 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":259,"threads_started":1}
I20260807 04:13:05.231750 3025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:05.230970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:49675)
0807 04:13:05.231494 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:05.231580 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":259,"threads_started":1}
I20260807 04:13:05.305277 3026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:05.304343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:46537)
0807 04:13:05.304960 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:05.305066 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":336,"threads_started":1}
I20260807 04:13:05.769178 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:05.768382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:43043)
0807 04:13:05.768907 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:05.768990 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":318,"threads_started":1}
I20260807 04:13:06.233683 3028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:06.232803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:48041)
0807 04:13:06.233417 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:06.233498 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":323,"threads_started":1}
I20260807 04:13:06.307008 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:06.306242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:35857)
0807 04:13:06.306741 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:06.306829 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":263,"threads_started":1}
I20260807 04:13:06.770947 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:06.770149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:56805)
0807 04:13:06.770639 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:06.770728 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":246,"threads_started":1}
I20260807 04:13:07.235296 3031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:07.234563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:34865)
0807 04:13:07.235036 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:07.235121 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":233,"threads_started":1}
I20260807 04:13:07.308925 3032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:07.308048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:52983)
0807 04:13:07.308647 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:07.308740 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":267,"threads_started":1}
I20260807 04:13:07.772948 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:07.772023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:43835)
0807 04:13:07.772663 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:07.772760 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":258,"threads_started":1}
I20260807 04:13:08.237511 3035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:08.236510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:50297)
0807 04:13:08.237167 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:08.237279 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":263,"threads_started":1}
I20260807 04:13:08.310716 3036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:08.309964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:60895)
0807 04:13:08.310448 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:08.310549 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":253,"threads_started":1}
I20260807 04:13:08.774706 3037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:08.773932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:58729)
0807 04:13:08.774441 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:08.774535 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":276,"threads_started":1}
I20260807 04:13:09.239179 3038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:09.238428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:36165)
0807 04:13:09.238905 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:09.239012 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":249,"threads_started":1}
I20260807 04:13:09.312712 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:09.311755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:50781)
0807 04:13:09.312303 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:09.312404 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":268,"threads_started":1}
I20260807 04:13:09.776846 3040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:09.775831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:53137)
0807 04:13:09.776463 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:09.776601 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":243,"threads_started":1}
I20260807 04:13:10.241201 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:10.240247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:53257)
0807 04:13:10.240889 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:10.240978 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":365,"threads_started":1}
I20260807 04:13:10.314474 3042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:10.313715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:53947)
0807 04:13:10.314189 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:10.314277 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":251,"threads_started":1}
I20260807 04:13:10.778779 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:10.777916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:60131)
0807 04:13:10.778535 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:10.778613 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":305,"threads_started":1}
I20260807 04:13:11.242964 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:11.242229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:48863)
0807 04:13:11.242704 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:11.242784 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":262,"threads_started":1}
I20260807 04:13:11.316252 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:11.315440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:54955)
0807 04:13:11.315934 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:11.316056 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":235,"threads_started":1}
I20260807 04:13:11.780540 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:11.779761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:56981)
0807 04:13:11.780267 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:11.780350 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":252,"threads_started":1}
I20260807 04:13:12.244791 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:12.243965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:38197)
0807 04:13:12.244457 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:12.244547 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":229,"threads_started":1}
I20260807 04:13:12.318428 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:12.317507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:53189)
0807 04:13:12.318060 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:12.318184 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":310,"threads_started":1}
I20260807 04:13:12.782647 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:12.781804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:42583)
0807 04:13:12.782391 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:12.782480 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":361,"threads_started":1}
I20260807 04:13:13.246850 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:13.245867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:48659)
0807 04:13:13.246597 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:13.246681 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":305,"threads_started":1}
I20260807 04:13:13.320324 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:13.319541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:60465)
0807 04:13:13.320043 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:13.320132 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":271,"threads_started":1}
I20260807 04:13:13.784626 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:13.783771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:39299)
0807 04:13:13.784276 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:13.784374 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":263,"threads_started":1}
I20260807 04:13:14.248795 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:14.247885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:55501)
0807 04:13:14.248328 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:14.248459 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":218,"threads_started":1}
I20260807 04:13:14.322279 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:14.321390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:39163)
0807 04:13:14.322023 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:14.322112 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":302,"threads_started":1}
I20260807 04:13:14.786438 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:14.785686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:59629)
0807 04:13:14.786161 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:14.786252 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":250,"threads_started":1}
I20260807 04:13:15.250736 3057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:15.249965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:50203)
0807 04:13:15.250478 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:15.250566 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
I20260807 04:13:15.324131 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:15.323277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:44295)
0807 04:13:15.323831 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:15.323940 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":292,"threads_started":1}
I20260807 04:13:15.788230 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:15.787429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:33797)
0807 04:13:15.787942 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:15.788034 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":244,"threads_started":1}
I20260807 04:13:16.252514 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:16.251696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:59469)
0807 04:13:16.252180 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:16.252291 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":260,"threads_started":1}
I20260807 04:13:16.325994 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:16.325148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:60209)
0807 04:13:16.325623 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:16.325712 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":231,"threads_started":1}
I20260807 04:13:16.790112 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:16.789312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:55613)
0807 04:13:16.789814 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:16.789919 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":254,"threads_started":1}
I20260807 04:13:17.254240 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:17.253500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:42933)
0807 04:13:17.253979 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:17.254058 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":237,"threads_started":1}
I20260807 04:13:17.328218 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:17.327250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:58567)
0807 04:13:17.327807 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:17.327947 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":280,"threads_started":1}
I20260807 04:13:17.792462 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:17.791498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.2:57283)
0807 04:13:17.792041 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:17.792184 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":289,"threads_started":1}
I20260807 04:13:18.239850 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 2558
I20260807 04:13:18.256269 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:18.255425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.3:40047)
0807 04:13:18.255966 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:18.256083 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":296,"threads_started":1}
I20260807 04:13:18.330276 3069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:18.329416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38421 (local address 127.18.193.1:33411)
0807 04:13:18.330020 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:18.330112 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":318,"threads_started":1}
I20260807 04:13:18.367234 3067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:18.241210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.18.193.1:38059)
0807 04:13:18.241758 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:18.241773 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:18.349357 (+107584us) server_negotiation.cc:407] Connection header received
0807 04:13:18.351342 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:18.351353 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:18.351810 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:18.352173 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:18.353796 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:18.355031 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:18.356286 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:18.357080 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:18.357305 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:18.360426 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:18.360435 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:18.360450 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:18.360493 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:18.363610 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:18.364386 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:18.364395 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:18.364412 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:18.364659 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:18.365079 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:18.365084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:18.365088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:18.365646 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:18.365988 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:18.366453 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 04:13:18.366911 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":254,"threads_started":1}
I20260807 04:13:18.368891 2726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:18.455132 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 2558
I20260807 04:13:18.474874 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 2734
I20260807 04:13:18.495149 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:18.475960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.18.193.2:36801)
0807 04:13:18.476752 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:18.476767 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:18.476806 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:13:18.478834 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:18.478845 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:18.479277 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:18.479701 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:18.481696 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:18.482997 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:18.484607 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:18.485294 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:18.485540 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:18.488482 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:18.488490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:18.488505 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:18.488545 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:18.491645 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:18.492362 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:18.492383 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:18.492389 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:18.492629 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:18.493027 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:18.493034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:18.493037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:18.493550 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:18.493875 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:18.494348 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 04:13:18.494777 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":424,"threads_started":1}
I20260807 04:13:18.496876 2871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:18.581939 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 2734
I20260807 04:13:18.601856 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 2879
I20260807 04:13:18.620749 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:18.602874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.18.193.3:33851)
0807 04:13:18.603530 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:18.603545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:18.603588 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:13:18.605331 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:18.605342 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:18.605770 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:18.606079 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:18.607563 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:18.608901 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:18.610381 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:18.611118 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:18.611280 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:18.614033 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:18.614041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:18.614057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:18.614099 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:18.617487 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:18.618173 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:18.618180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:18.618187 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:18.618355 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:18.618710 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:18.618713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:18.618716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:18.619280 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:18.619532 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:18.619919 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 04:13:18.620276 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":335,"threads_started":1}
I20260807 04:13:18.622215 3011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:18.706799 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 2879
2026-08-07T04:13:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24924 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:13:18 dist-test-slave-8n49 krb5kdc[3077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:13:18 dist-test-slave-8n49 krb5kdc[3077](info): set up 2 sockets
Aug 07 04:13:18 dist-test-slave-8n49 krb5kdc[3077](info): commencing operation
krb5kdc: starting...
W20260807 04:13:19.860090 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:13:20 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:13:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:13:20.144833 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41531
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:41531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:13:20.489917 3091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:20.490233 3091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:20.490329 3091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:20.497747 3091 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 04:13:20.497859 3091 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 04:13:20.497897 3091 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 04:13:20.497943 3091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:13:20.497999 3091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:20.498075 3091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:20.498121 3091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:13:20.498171 3091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:13:20.508317 3091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:41531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:20.510075 3091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:20.511476 3091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:20.524205 3096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:20.524248 3099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:20.524174 3097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:20.524610 3091 server_base.cc:1034] running on GCE node
I20260807 04:13:20.525647 3091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:20.527066 3091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:20.528374 3091 hybrid_clock.cc:648] HybridClock initialized: now 1786076000528293 us; error 50 us; skew 500 ppm
I20260807 04:13:20.528822 3091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:20 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076000, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:20.535561 3091 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:13:20.537420 3091 webserver.cc:460] Webserver started at http://127.18.193.62:39825/ using document root <none> and password file <none>
I20260807 04:13:20.538211 3091 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:20.538362 3091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:20.538731 3091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:13:20.542171 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "56908b36ebe2410abc5ff3e990d694e3"
format_stamp: "Formatted at 2026-08-07 04:13:20 on dist-test-slave-8n49"
server_key: "940e6b5623e5ae1eebcb16280052f255"
server_key_iv: "047c3be58ddf35a854a4c535f46eb504"
server_key_version: "encryptionkey@0"
I20260807 04:13:20.543056 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "56908b36ebe2410abc5ff3e990d694e3"
format_stamp: "Formatted at 2026-08-07 04:13:20 on dist-test-slave-8n49"
server_key: "940e6b5623e5ae1eebcb16280052f255"
server_key_iv: "047c3be58ddf35a854a4c535f46eb504"
server_key_version: "encryptionkey@0"
I20260807 04:13:20.548914 3091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 04:13:20.552731 3107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:20.554232 3091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:13:20.554440 3091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "56908b36ebe2410abc5ff3e990d694e3"
format_stamp: "Formatted at 2026-08-07 04:13:20 on dist-test-slave-8n49"
server_key: "940e6b5623e5ae1eebcb16280052f255"
server_key_iv: "047c3be58ddf35a854a4c535f46eb504"
server_key_version: "encryptionkey@0"
I20260807 04:13:20.554723 3091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:20.572635 3091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:20.576864 3091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:20.577140 3091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:20.596117 3091 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:41531
I20260807 04:13:20.596136 3162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:41531 every 8 connection(s)
I20260807 04:13:20.597563 3091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:13:20.601369 3163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:20.603310 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 3091
I20260807 04:13:20.603579 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:13:20.604282 19204 external_mini_cluster.cc:1428] Setting key be24417c09cf8434c1e13c022a78d87f
I20260807 04:13:20.615049 3163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Bootstrap starting.
I20260807 04:13:20.618906 3163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:20.620021 3163 log.cc:824] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:13:20 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:20.623063 3163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: No bootstrap required, opened a new log
I20260807 04:13:20.629541 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:20.606436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.18.193.62:41531)
0807 04:13:20.607364 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:20.607380 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:20.607444 (+ 64us) server_negotiation.cc:407] Connection header received
0807 04:13:20.609874 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:20.609903 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:20.610439 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:20.610899 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:20.612389 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:20.614437 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:20.615921 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:20.617061 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:20.617264 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:20.621791 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:20.621798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:20.621814 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:20.621848 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:20.625085 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:20.626165 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:20.626171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:20.626177 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:20.626343 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:20.626750 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:20.626753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:20.626755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:20.627312 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:20.627546 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:20.628036 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 04:13:20.628429 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":375,"threads_started":1}
I20260807 04:13:20.632599 3163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:20.633078 3163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:20.633188 3163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56908b36ebe2410abc5ff3e990d694e3, State: Initialized, Role: FOLLOWER
I20260807 04:13:20.633741 3163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:20.633985 3163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:13:20.634119 3163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:13:20.634294 3163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:20.636516 3163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:20.637226 3163 leader_election.cc:302] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56908b36ebe2410abc5ff3e990d694e3; no voters:
I20260807 04:13:20.637854 3163 leader_election.cc:288] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:13:20.638038 3168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:13:20.639362 3168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 LEADER]: Becoming Leader. State: Replica: 56908b36ebe2410abc5ff3e990d694e3, State: Running, Role: LEADER
I20260807 04:13:20.639952 3168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:20.640201 3163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:13:20.643100 3169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56908b36ebe2410abc5ff3e990d694e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } } }
I20260807 04:13:20.643532 3169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:13:20.643988 3170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56908b36ebe2410abc5ff3e990d694e3. Latest consensus state: current_term: 1 leader_uuid: "56908b36ebe2410abc5ff3e990d694e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } } }
I20260807 04:13:20.644338 3170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:13:20.653249 3176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:13:20.658870 3176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:13:20.668393 3176 catalog_manager.cc:1347] Generated new cluster ID: 19970674dbca4aeabef1db6157099964
I20260807 04:13:20.668635 3176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:13:20.691761 3176 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:13:20.693780 3176 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:13:20.701217 3176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Generated new TSK 0
I20260807 04:13:20.702203 3176 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:13:20.769053 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41531
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:13:21.113521 3190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:21.113868 3190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:21.114046 3190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:21.121843 3190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:21.121995 3190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:21.122174 3190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:13:21.132104 3190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:21.133456 3190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:21.134752 3190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:21.148284 3200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:21.148366 3196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:21.148257 3198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:21.149025 3190 server_base.cc:1034] running on GCE node
I20260807 04:13:21.149699 3190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:21.150661 3190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:21.151885 3190 hybrid_clock.cc:648] HybridClock initialized: now 1786076001151821 us; error 65 us; skew 500 ppm
I20260807 04:13:21.152202 3190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:21 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076001, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:21.155483 3190 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:13:21.157006 3190 webserver.cc:460] Webserver started at http://127.18.193.1:40387/ using document root <none> and password file <none>
I20260807 04:13:21.157769 3190 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:21.157912 3190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:21.158260 3190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:13:21.161350 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "9c63e16bfee648fe90400cdff5781fb1"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "3d51b549ecbddb93de819667ee4ddbc8"
server_key_iv: "daec79c24691f22575391899c6f5f5e0"
server_key_version: "encryptionkey@0"
I20260807 04:13:21.162111 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "9c63e16bfee648fe90400cdff5781fb1"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "3d51b549ecbddb93de819667ee4ddbc8"
server_key_iv: "daec79c24691f22575391899c6f5f5e0"
server_key_version: "encryptionkey@0"
I20260807 04:13:21.167243 3190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:13:21.170861 3207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:21.172264 3190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:13:21.172505 3190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "9c63e16bfee648fe90400cdff5781fb1"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "3d51b549ecbddb93de819667ee4ddbc8"
server_key_iv: "daec79c24691f22575391899c6f5f5e0"
server_key_version: "encryptionkey@0"
I20260807 04:13:21.172799 3190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:21.201793 3190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:21.206432 3190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:21.206733 3190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:21.207679 3190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:21.209075 3190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:21.209179 3190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:21.209331 3190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:21.209412 3190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:21.243453 3190 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:36177
I20260807 04:13:21.243500 3326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:36177 every 8 connection(s)
I20260807 04:13:21.244814 3190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:13:21.249543 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 3190
I20260807 04:13:21.249760 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:13:21.250458 19204 external_mini_cluster.cc:1428] Setting key 177b9f63c697f1b9f4abbc4dc467f1e2
Aug 07 04:13:21 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076001, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:21.267869 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:21.247652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:59547 (local address 127.18.193.62:41531)
0807 04:13:21.248181 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:21.248186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:21.250999 (+ 2813us) server_negotiation.cc:407] Connection header received
0807 04:13:21.251114 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:21.251119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:21.251186 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:21.251331 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:21.253263 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.254427 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.256029 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.256779 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.256928 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:21.260972 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:21.260980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:21.260988 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:21.261044 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:21.264291 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:21.265399 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:21.265403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:21.265406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:21.265539 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:21.266070 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:21.266073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:21.266075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:21.266344 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:21.266457 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:21.267424 (+ 967us) server_negotiation.cc:299] Negotiation successful
0807 04:13:21.267606 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":285,"threads_started":1}
I20260807 04:13:21.269595 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:21.248211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:59547)
0807 04:13:21.248962 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:21.249000 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:21.250842 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:21.251585 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:21.251596 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:21.252329 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:13:21.253085 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:21.253103 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.254595 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.254599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:21.255864 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:21.255876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.257000 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.257006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:21.257201 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:21.257945 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:13:21.257969 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:13:21.260734 (+ 2765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:13:21.264478 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:21.264491 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:21.264512 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:21.265224 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:21.265663 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:21.265671 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:21.265678 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:21.265936 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:21.266485 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:13:21.266496 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:13:21.266995 (+ 499us) client_negotiation.cc:769] Sending connection context
0807 04:13:21.267337 (+ 342us) client_negotiation.cc:240] Negotiation successful
0807 04:13:21.267987 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":335,"threads_started":1}
I20260807 04:13:21.271497 3327 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41531
I20260807 04:13:21.271842 3327 heartbeater.cc:464] Registering TS with master...
I20260807 04:13:21.272719 3327 heartbeater.cc:511] Master 127.18.193.62:41531 requested a full tablet report, sending...
I20260807 04:13:21.275209 3125 ts_manager.cc:194] Registered new tserver with Master: 9c63e16bfee648fe90400cdff5781fb1 (127.18.193.1:36177)
I20260807 04:13:21.277547 3125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:59547
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:13:21.316360 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41531
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:13:21.652966 3335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:21.653251 3335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:21.653432 3335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:21.660959 3335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:21.661077 3335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:21.661293 3335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:13:21.671355 3335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:21.672798 3335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:21.674053 3335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:21.687613 3340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:21.687719 3343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:21.687664 3341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:21.688591 3335 server_base.cc:1034] running on GCE node
I20260807 04:13:21.689217 3335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:21.690162 3335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:21.691408 3335 hybrid_clock.cc:648] HybridClock initialized: now 1786076001691367 us; error 52 us; skew 500 ppm
I20260807 04:13:21.691771 3335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:21 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076001, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:21.694988 3335 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:13:21.696442 3335 webserver.cc:460] Webserver started at http://127.18.193.2:35947/ using document root <none> and password file <none>
I20260807 04:13:21.697173 3335 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:21.697283 3335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:21.697546 3335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:13:21.700403 3335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "2412993129d64d75ad7bc8f852bbc0e5"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "192450a0d597abaf756f1aaf34a34d81"
server_key_iv: "4780b4b7ed43eb5bf1a409c629a68b07"
server_key_version: "encryptionkey@0"
I20260807 04:13:21.701210 3335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "2412993129d64d75ad7bc8f852bbc0e5"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "192450a0d597abaf756f1aaf34a34d81"
server_key_iv: "4780b4b7ed43eb5bf1a409c629a68b07"
server_key_version: "encryptionkey@0"
I20260807 04:13:21.706408 3335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:13:21.709726 3350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:21.711117 3335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:13:21.711261 3335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "2412993129d64d75ad7bc8f852bbc0e5"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "192450a0d597abaf756f1aaf34a34d81"
server_key_iv: "4780b4b7ed43eb5bf1a409c629a68b07"
server_key_version: "encryptionkey@0"
I20260807 04:13:21.711423 3335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:21.725996 3335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:21.730113 3335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:21.730358 3335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:21.731274 3335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:21.732664 3335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:21.732743 3335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:21.732828 3335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:21.732877 3335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:21.766531 3335 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:45411
I20260807 04:13:21.766584 3472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:45411 every 8 connection(s)
I20260807 04:13:21.767841 3335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:13:21.773694 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 3335
I20260807 04:13:21.773921 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:13:21.774462 19204 external_mini_cluster.cc:1428] Setting key 330e7a8affbd81855f4530851e8967ab
Aug 07 04:13:21 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076001, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:21.788178 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:21.770377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:40179 (local address 127.18.193.62:41531)
0807 04:13:21.770836 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:21.770841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:21.771771 (+ 930us) server_negotiation.cc:407] Connection header received
0807 04:13:21.773884 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:21.773894 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:21.773990 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:21.774140 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:21.775812 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.776781 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.778409 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.778830 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.778975 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:21.782403 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:21.782407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:21.782411 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:21.782442 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:21.785103 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:21.785838 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:21.785845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:21.785849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:21.785981 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:21.786438 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:21.786444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:21.786447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:21.786718 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:21.786878 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:21.787554 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 04:13:21.787781 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":219,"threads_started":1}
I20260807 04:13:21.789216 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:21.770931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:40179)
0807 04:13:21.771616 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:21.771652 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:21.773552 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:21.774376 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:21.774384 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:21.774932 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:13:21.775645 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:21.775665 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.776949 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.776954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:21.778247 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:21.778259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:21.779055 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:21.779062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:21.779293 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:21.780001 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:13:21.780022 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:13:21.782168 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:13:21.785249 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:21.785259 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:21.785276 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:21.785712 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:21.786114 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:21.786119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:21.786123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:21.786321 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:21.786849 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:13:21.786856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:13:21.787194 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 04:13:21.787464 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 04:13:21.788025 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":265,"threads_started":1}
I20260807 04:13:21.790709 3473 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41531
I20260807 04:13:21.791005 3473 heartbeater.cc:464] Registering TS with master...
I20260807 04:13:21.791728 3473 heartbeater.cc:511] Master 127.18.193.62:41531 requested a full tablet report, sending...
I20260807 04:13:21.793395 3125 ts_manager.cc:194] Registered new tserver with Master: 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411)
I20260807 04:13:21.794678 3125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:40179
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:13:21.836506 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41531
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:13:22.178006 3479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:22.178344 3479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:22.178443 3479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:22.185762 3479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:22.185868 3479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:22.185982 3479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:13:22.196100 3479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:22.197443 3479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:22.198704 3479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:22.211920 3488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:22.212008 3484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:22.211952 3485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:22.212859 3479 server_base.cc:1034] running on GCE node
I20260807 04:13:22.213469 3479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:22.214397 3479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:22.215646 3479 hybrid_clock.cc:648] HybridClock initialized: now 1786076002215586 us; error 61 us; skew 500 ppm
I20260807 04:13:22.216022 3479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:22 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076002, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:22.219405 3479 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:13:22.221071 3479 webserver.cc:460] Webserver started at http://127.18.193.3:42517/ using document root <none> and password file <none>
I20260807 04:13:22.221782 3479 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:22.221935 3479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:22.222249 3479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:13:22.225288 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "9db9764d15a0471f9f2830e8c45237c3"
format_stamp: "Formatted at 2026-08-07 04:13:22 on dist-test-slave-8n49"
server_key: "a3569a85d330a6b99a7c98ad0b0b1545"
server_key_iv: "aac77176d9961b24eb7dd895845cbef4"
server_key_version: "encryptionkey@0"
I20260807 04:13:22.226079 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "9db9764d15a0471f9f2830e8c45237c3"
format_stamp: "Formatted at 2026-08-07 04:13:22 on dist-test-slave-8n49"
server_key: "a3569a85d330a6b99a7c98ad0b0b1545"
server_key_iv: "aac77176d9961b24eb7dd895845cbef4"
server_key_version: "encryptionkey@0"
I20260807 04:13:22.231350 3479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:13:22.235056 3495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:22.236392 3479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:13:22.236652 3479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "9db9764d15a0471f9f2830e8c45237c3"
format_stamp: "Formatted at 2026-08-07 04:13:22 on dist-test-slave-8n49"
server_key: "a3569a85d330a6b99a7c98ad0b0b1545"
server_key_iv: "aac77176d9961b24eb7dd895845cbef4"
server_key_version: "encryptionkey@0"
I20260807 04:13:22.236865 3479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:22.260349 3479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:22.264972 3479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:22.265275 3479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:22.266299 3479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:22.267637 3479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:22.267748 3479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:22.267902 3479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:22.267983 3479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:22.280773 3327 heartbeater.cc:503] Master 127.18.193.62:41531 was elected leader, sending a full tablet report...
I20260807 04:13:22.303079 3479 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:32777
I20260807 04:13:22.303169 3615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:32777 every 8 connection(s)
I20260807 04:13:22.304399 3479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:13:22.307816 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 3479
I20260807 04:13:22.308033 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:13:22.308533 19204 external_mini_cluster.cc:1428] Setting key 897cb0aff91a8c93b056b28721213f6f
Aug 07 04:13:22 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076002, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:22.325173 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.307128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:57835 (local address 127.18.193.62:41531)
0807 04:13:22.307719 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.307726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.308641 (+ 915us) server_negotiation.cc:407] Connection header received
0807 04:13:22.310917 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.310922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.311016 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.311140 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:22.312744 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.313536 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.315141 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.315612 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.315750 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.319420 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:22.319425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:22.319428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:22.319460 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:22.322029 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.322769 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.322779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.322782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.322904 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.323359 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.323363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.323365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.323648 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:22.323837 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.324516 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.324780 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":284,"threads_started":1}
I20260807 04:13:22.326110 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.307622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:57835)
0807 04:13:22.308441 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:22.308487 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:22.310595 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:22.311399 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:22.311408 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:22.311922 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:13:22.312515 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.312537 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.313702 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.313715 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.314990 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.315003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.315845 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.315854 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.316162 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.316908 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:13:22.316931 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:13:22.319196 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:13:22.322184 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:22.322193 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:22.322210 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:22.322635 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:22.323020 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:22.323025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:22.323029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:22.323223 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:22.323785 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:13:22.323795 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:13:22.324142 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 04:13:22.324407 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 04:13:22.324963 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":354,"threads_started":1}
I20260807 04:13:22.327455 3616 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41531
I20260807 04:13:22.327731 3616 heartbeater.cc:464] Registering TS with master...
I20260807 04:13:22.328403 3616 heartbeater.cc:511] Master 127.18.193.62:41531 requested a full tablet report, sending...
I20260807 04:13:22.330039 3125 ts_manager.cc:194] Registered new tserver with Master: 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777)
I20260807 04:13:22.330817 3125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:57835
I20260807 04:13:22.336287 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:13:22.354151 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.341548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.18.193.62:41531)
0807 04:13:22.341766 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.341771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.342249 (+ 478us) server_negotiation.cc:407] Connection header received
0807 04:13:22.342369 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.342373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.342439 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.342557 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:22.344000 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.344849 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.346316 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.346739 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.346840 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.349287 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:22.349292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:22.349296 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:22.349327 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:22.351867 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.352520 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.352524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.352526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.352651 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.352967 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.352970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.352972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.353273 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:22.353499 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.353754 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.353914 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 04:13:22.358114 3125 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:13:22.365206 3125 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:13:22.405205 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.379395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.18.193.2:45411)
0807 04:13:22.396424 (+ 17029us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.396434 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.396470 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:13:22.396621 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.396630 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.396894 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.397103 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:22.398152 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.399683 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.402137 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.403651 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.403790 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.404036 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.404658 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.404842 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16881,"thread_start_us":16770,"threads_started":1}
I20260807 04:13:22.405252 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.378930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45411 (local address 127.0.0.1:34534)
0807 04:13:22.395605 (+ 16675us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:22.395644 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:22.395823 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:22.397410 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:22.397418 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:22.397445 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:22.397918 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.397936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.399821 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.399829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.401930 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.401944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.404168 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.404175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.404368 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.404429 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 04:13:22.404595 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 04:13:22.404798 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16425,"thread_start_us":19653,"threads_started":1}
I20260807 04:13:22.407066 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.378928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:32777 (local address 127.0.0.1:42048)
0807 04:13:22.398860 (+ 19932us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:22.398876 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:22.399028 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:22.399891 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:22.399895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:22.399923 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:22.400386 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.400406 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.402099 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.402104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.403969 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.403982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.406183 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.406189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.406349 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.406384 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:13:22.406502 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 04:13:22.406637 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19816,"mutex_wait_us":680,"thread_start_us":15584,"threads_started":1}
I20260807 04:13:22.407083 3632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.379418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42048 (local address 127.18.193.3:32777)
0807 04:13:22.394784 (+ 15366us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.394797 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.399013 (+ 4216us) server_negotiation.cc:407] Connection header received
0807 04:13:22.399190 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.399195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.399447 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.399666 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:22.400632 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.401942 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.404138 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.405980 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.406124 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.406297 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.406575 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.406751 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15123,"thread_start_us":15395,"threads_started":1}
I20260807 04:13:22.407584 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.380211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.18.193.1:36177)
0807 04:13:22.397975 (+ 17764us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.397984 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.398740 (+ 756us) server_negotiation.cc:407] Connection header received
0807 04:13:22.398844 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.398851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.399079 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.399288 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:22.399925 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.401332 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.403947 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.405456 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.405605 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.405841 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.407178 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.407359 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16523,"thread_start_us":16383,"threads_started":1}
I20260807 04:13:22.407595 3633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.379558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:44954)
0807 04:13:22.398451 (+ 18893us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:22.398473 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:22.398598 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:22.399275 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:22.399279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:22.399303 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:22.399705 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.399722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.401493 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.401498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.403759 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.403773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.406424 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.406429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.406624 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.406656 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:13:22.406750 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 04:13:22.406884 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18757,"thread_start_us":18573,"threads_started":1}
I20260807 04:13:22.409595 3401 tablet_service.cc:1467] Processing CreateTablet for tablet 0e46f42ea6f04852853ac0e3a3e07c8b (DEFAULT_TABLE table=test-table [id=12cc47877aef493ab8e26ac7a2c8a136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:22.411175 3401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e46f42ea6f04852853ac0e3a3e07c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:22.411295 3259 tablet_service.cc:1467] Processing CreateTablet for tablet 0e46f42ea6f04852853ac0e3a3e07c8b (DEFAULT_TABLE table=test-table [id=12cc47877aef493ab8e26ac7a2c8a136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:22.411295 3547 tablet_service.cc:1467] Processing CreateTablet for tablet 0e46f42ea6f04852853ac0e3a3e07c8b (DEFAULT_TABLE table=test-table [id=12cc47877aef493ab8e26ac7a2c8a136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:22.412753 3259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e46f42ea6f04852853ac0e3a3e07c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:22.412770 3547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e46f42ea6f04852853ac0e3a3e07c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:22.422880 3636 tablet_bootstrap.cc:492] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Bootstrap starting.
I20260807 04:13:22.426076 3636 tablet_bootstrap.cc:654] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:22.427292 3638 tablet_bootstrap.cc:492] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Bootstrap starting.
I20260807 04:13:22.427350 3636 log.cc:824] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:22.428224 3637 tablet_bootstrap.cc:492] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Bootstrap starting.
I20260807 04:13:22.432672 3637 tablet_bootstrap.cc:654] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:22.434290 3637 log.cc:824] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:22.435853 3638 tablet_bootstrap.cc:654] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:22.437216 3638 log.cc:824] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:22.438015 3636 tablet_bootstrap.cc:492] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: No bootstrap required, opened a new log
I20260807 04:13:22.438040 3637 tablet_bootstrap.cc:492] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: No bootstrap required, opened a new log
I20260807 04:13:22.438318 3636 ts_tablet_manager.cc:1397] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260807 04:13:22.438323 3637 ts_tablet_manager.cc:1397] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260807 04:13:22.440582 3638 tablet_bootstrap.cc:492] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: No bootstrap required, opened a new log
I20260807 04:13:22.440959 3638 ts_tablet_manager.cc:1397] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260807 04:13:22.446839 3637 raft_consensus.cc:348] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.447237 3637 raft_consensus.cc:374] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:22.447389 3637 raft_consensus.cc:729] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db9764d15a0471f9f2830e8c45237c3, State: Initialized, Role: FOLLOWER
I20260807 04:13:22.448068 3637 consensus_queue.cc:260] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.449222 3616 heartbeater.cc:503] Master 127.18.193.62:41531 was elected leader, sending a full tablet report...
I20260807 04:13:22.449501 3636 raft_consensus.cc:348] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.449951 3637 ts_tablet_manager.cc:1428] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260807 04:13:22.449958 3636 raft_consensus.cc:374] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:22.450055 3636 raft_consensus.cc:729] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2412993129d64d75ad7bc8f852bbc0e5, State: Initialized, Role: FOLLOWER
I20260807 04:13:22.450657 3636 consensus_queue.cc:260] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.451886 3473 heartbeater.cc:503] Master 127.18.193.62:41531 was elected leader, sending a full tablet report...
I20260807 04:13:22.451730 3638 raft_consensus.cc:348] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.452148 3638 raft_consensus.cc:374] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:22.452243 3638 raft_consensus.cc:729] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c63e16bfee648fe90400cdff5781fb1, State: Initialized, Role: FOLLOWER
I20260807 04:13:22.452628 3636 ts_tablet_manager.cc:1428] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260807 04:13:22.452993 3638 consensus_queue.cc:260] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.454690 3638 ts_tablet_manager.cc:1428] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260807 04:13:22.497489 3328 tablet.cc:2369] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:13:22.519974 3474 tablet.cc:2369] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:13:22.556134 3617 tablet.cc:2369] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:13:22.559468 3645 raft_consensus.cc:482] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:13:22.559885 3645 raft_consensus.cc:504] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.561767 3645 leader_election.cc:288] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777), 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411)
I20260807 04:13:22.569965 3632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.562115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:59889 (local address 127.18.193.3:32777)
0807 04:13:22.562357 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.562364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.563434 (+ 1070us) server_negotiation.cc:407] Connection header received
0807 04:13:22.563686 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.563691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.563762 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.563864 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:22.564413 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.565225 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.567437 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.568698 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.568872 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.568955 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.569552 (+ 597us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.569684 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:13:22.570109 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.562405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45411 (local address 127.18.193.1:39359)
0807 04:13:22.563522 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:22.563536 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:22.563697 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:22.564049 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:22.564054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:22.564082 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:22.564486 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.564495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.565545 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.565552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.567398 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.567419 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.569024 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.569028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.569303 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.569398 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 04:13:22.569507 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:13:22.569694 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":89,"thread_start_us":657,"threads_started":1}
I20260807 04:13:22.570504 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.562342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:32777 (local address 127.18.193.1:59889)
0807 04:13:22.563317 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:22.563333 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:22.563527 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:22.563881 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:22.563885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:22.563910 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:22.564264 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.564275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.565358 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.565365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.567188 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:22.567204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.569001 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.569012 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:22.569305 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.569403 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 04:13:22.569507 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 04:13:22.570020 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":153,"thread_start_us":492,"threads_started":1}
I20260807 04:13:22.570519 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.562469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:39359 (local address 127.18.193.2:45411)
0807 04:13:22.562732 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.562740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.563632 (+ 892us) server_negotiation.cc:407] Connection header received
0807 04:13:22.563840 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.563845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.563922 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.564025 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:22.564669 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.565381 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.567556 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.568811 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.568952 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.569021 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.569566 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.569689 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 04:13:22.571579 3424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e46f42ea6f04852853ac0e3a3e07c8b" candidate_uuid: "9c63e16bfee648fe90400cdff5781fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2412993129d64d75ad7bc8f852bbc0e5" is_pre_election: true
I20260807 04:13:22.571722 3568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e46f42ea6f04852853ac0e3a3e07c8b" candidate_uuid: "9c63e16bfee648fe90400cdff5781fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db9764d15a0471f9f2830e8c45237c3" is_pre_election: true
I20260807 04:13:22.572171 3424 raft_consensus.cc:2454] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c63e16bfee648fe90400cdff5781fb1 in term 0.
I20260807 04:13:22.572252 3568 raft_consensus.cc:2454] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c63e16bfee648fe90400cdff5781fb1 in term 0.
I20260807 04:13:22.572957 3209 leader_election.cc:302] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2412993129d64d75ad7bc8f852bbc0e5, 9c63e16bfee648fe90400cdff5781fb1; no voters:
I20260807 04:13:22.573427 3645 raft_consensus.cc:2781] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:13:22.573603 3645 raft_consensus.cc:482] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:13:22.573720 3645 raft_consensus.cc:3037] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:22.576001 3645 raft_consensus.cc:504] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.576640 3645 leader_election.cc:288] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 1 election: Requested vote from peers 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777), 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411)
I20260807 04:13:22.577185 3424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e46f42ea6f04852853ac0e3a3e07c8b" candidate_uuid: "9c63e16bfee648fe90400cdff5781fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2412993129d64d75ad7bc8f852bbc0e5"
I20260807 04:13:22.577185 3568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e46f42ea6f04852853ac0e3a3e07c8b" candidate_uuid: "9c63e16bfee648fe90400cdff5781fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db9764d15a0471f9f2830e8c45237c3"
I20260807 04:13:22.577405 3568 raft_consensus.cc:3037] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:22.577405 3424 raft_consensus.cc:3037] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:22.580663 3568 raft_consensus.cc:2454] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c63e16bfee648fe90400cdff5781fb1 in term 1.
I20260807 04:13:22.580663 3424 raft_consensus.cc:2454] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c63e16bfee648fe90400cdff5781fb1 in term 1.
I20260807 04:13:22.581063 3209 leader_election.cc:302] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2412993129d64d75ad7bc8f852bbc0e5, 9c63e16bfee648fe90400cdff5781fb1; no voters:
I20260807 04:13:22.581372 3645 raft_consensus.cc:2781] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:13:22.581817 3645 raft_consensus.cc:686] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 1 LEADER]: Becoming Leader. State: Replica: 9c63e16bfee648fe90400cdff5781fb1, State: Running, Role: LEADER
I20260807 04:13:22.582305 3645 consensus_queue.cc:237] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } }
I20260807 04:13:22.586952 3125 catalog_manager.cc:5495] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c63e16bfee648fe90400cdff5781fb1 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "9c63e16bfee648fe90400cdff5781fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } health_report { overall_health: HEALTHY } } }
I20260807 04:13:22.630220 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.624710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.18.193.1:36177)
0807 04:13:22.624933 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.624939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.624970 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:13:22.625108 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.625111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.625190 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.625352 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:13:22.625762 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.626504 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.627941 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.628370 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.628512 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.629087 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:13:22.629618 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:13:22.629793 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.629843 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.629927 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:13:22.639195 3424 raft_consensus.cc:1261] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Refusing update from remote peer 9c63e16bfee648fe90400cdff5781fb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:13:22.639251 3568 raft_consensus.cc:1261] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 1 FOLLOWER]: Refusing update from remote peer 9c63e16bfee648fe90400cdff5781fb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:13:22.640272 3648 consensus_queue.cc:1035] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:13:22.640787 3645 consensus_queue.cc:1035] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:13:22.653239 3652 mvcc.cc:205] Tried to move back new op lower bound from 7315767306799411200 to 7315767306580860928. Current Snapshot: MvccSnapshot[applied={T|T < 7315767306799411200}]
I20260807 04:13:22.672312 3124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:13:22.672698 3124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:13:22.676450 3124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Sending DeleteTablet for 3 replicas of tablet 0e46f42ea6f04852853ac0e3a3e07c8b
I20260807 04:13:22.677731 3547 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e46f42ea6f04852853ac0e3a3e07c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:22 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:42048
I20260807 04:13:22.677804 3401 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e46f42ea6f04852853ac0e3a3e07c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:22 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:34534
I20260807 04:13:22.678797 3661 tablet_replica.cc:330] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: stopping tablet replica
I20260807 04:13:22.678791 3259 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e46f42ea6f04852853ac0e3a3e07c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:22 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:44954
I20260807 04:13:22.679478 3661 raft_consensus.cc:2229] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:13:22.679836 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 3190
I20260807 04:13:22.680235 3661 raft_consensus.cc:2258] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:22.683686 3661 ts_tablet_manager.cc:1905] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:22.692623 3663 tablet_replica.cc:330] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: stopping tablet replica
I20260807 04:13:22.693151 3663 raft_consensus.cc:2229] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:13:22.693611 3662 tablet_replica.cc:330] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: stopping tablet replica
I20260807 04:13:22.693895 3663 raft_consensus.cc:2258] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:22.694072 3662 raft_consensus.cc:2229] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:13:22.694633 3662 raft_consensus.cc:2258] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:22.696405 3663 ts_tablet_manager.cc:1905] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:22.696646 3662 ts_tablet_manager.cc:1905] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:22.704185 3663 ts_tablet_manager.cc:1918] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:13:22.704185 3662 ts_tablet_manager.cc:1918] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 07 04:13:22 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:13:22.704356 3662 log.cc:1196] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/0e46f42ea6f04852853ac0e3a3e07c8b
I20260807 04:13:22.704391 3663 log.cc:1196] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/0e46f42ea6f04852853ac0e3a3e07c8b
I20260807 04:13:22.704969 3662 ts_tablet_manager.cc:1939] T 0e46f42ea6f04852853ac0e3a3e07c8b P 2412993129d64d75ad7bc8f852bbc0e5: Deleting consensus metadata
I20260807 04:13:22.704978 3663 ts_tablet_manager.cc:1939] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9c63e16bfee648fe90400cdff5781fb1: Deleting consensus metadata
I20260807 04:13:22.707037 3108 catalog_manager.cc:4841] TS 9c63e16bfee648fe90400cdff5781fb1 (127.18.193.1:36177): tablet 0e46f42ea6f04852853ac0e3a3e07c8b (table test-table [id=12cc47877aef493ab8e26ac7a2c8a136]) successfully deleted
I20260807 04:13:22.707123 3109 catalog_manager.cc:4841] TS 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411): tablet 0e46f42ea6f04852853ac0e3a3e07c8b (table test-table [id=12cc47877aef493ab8e26ac7a2c8a136]) successfully deleted
I20260807 04:13:22.707610 3661 ts_tablet_manager.cc:1918] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:13:22.707810 3661 log.cc:1196] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/0e46f42ea6f04852853ac0e3a3e07c8b
I20260807 04:13:22.708367 3661 ts_tablet_manager.cc:1939] T 0e46f42ea6f04852853ac0e3a3e07c8b P 9db9764d15a0471f9f2830e8c45237c3: Deleting consensus metadata
I20260807 04:13:22.710402 3111 catalog_manager.cc:4841] TS 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777): tablet 0e46f42ea6f04852853ac0e3a3e07c8b (table test-table [id=12cc47877aef493ab8e26ac7a2c8a136]) successfully deleted
I20260807 04:13:22.713919 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.681319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.18.193.1:36177)
0807 04:13:22.681491 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.681497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.681519 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:13:22.681789 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.681794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.681894 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.682014 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:22.695724 (+ 13710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.696984 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.699058 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.699772 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.699874 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.706064 (+ 6190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:22.706081 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:22.706101 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:22.706173 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:22.710708 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.711705 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.711712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.711714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.711857 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.712297 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.712300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.712302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.712770 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:22.712993 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.713391 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.713604 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:13:22.714649 3322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:22.828970 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 3190
I20260807 04:13:22.850230 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 3335
Aug 07 04:13:22 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:13:22.867911 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.851068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.18.193.2:45411)
0807 04:13:22.851272 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.851280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.851372 (+ 92us) server_negotiation.cc:407] Connection header received
0807 04:13:22.851600 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.851604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.851689 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.851803 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:22.853463 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.854449 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.856250 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.856972 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.857124 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.861840 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:22.861858 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:22.861876 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:22.861919 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:22.864851 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.865635 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.865643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.865645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.865774 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.866183 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.866188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.866194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.866630 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:22.866953 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.867313 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.867548 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:13:22.868692 3467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:22.960273 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 3335
I20260807 04:13:22.979218 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 3479
Aug 07 04:13:22 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:13:22.995286 3632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:22.980099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42060 (local address 127.18.193.3:32777)
0807 04:13:22.980317 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:22.980324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:22.980348 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:13:22.980601 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:22.980609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:22.980693 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:22.980807 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:22.982252 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.982998 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.984390 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:22.984886 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:22.985036 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:22.989699 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:22.989717 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:22.989735 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:22.989778 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:22.992405 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.993286 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.993293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.993296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.993465 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:22.993798 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:22.993801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:22.993803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:22.994162 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:22.994380 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:22.994743 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 04:13:22.994915 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:13:22.996040 3610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:23.086153 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 3479
I20260807 04:13:23.104700 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 3091
I20260807 04:13:23.105911 3158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:23.216960 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 3091
I20260807 04:13:43.241786 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41531
--webserver_interface=127.18.193.62
--webserver_port=39825
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:41531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:13:43.614029 3669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:43.614338 3669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:43.614449 3669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:43.622519 3669 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 04:13:43.622627 3669 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 04:13:43.622668 3669 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 04:13:43.622714 3669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:13:43.622769 3669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:43.622839 3669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:43.622885 3669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:13:43.622939 3669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:13:43.633213 3669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:41531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=39825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:43.634582 3669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:43.635921 3669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:43.662662 3678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:43.662638 3675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:43.663087 3669 server_base.cc:1034] running on GCE node
W20260807 04:13:43.662593 3676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:43.664258 3669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:43.666002 3669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:43.667251 3669 hybrid_clock.cc:648] HybridClock initialized: now 1786076023667177 us; error 42 us; skew 500 ppm
I20260807 04:13:43.667680 3669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:43 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:43.679688 3669 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:13:43.681704 3669 webserver.cc:460] Webserver started at http://127.18.193.62:39825/ using document root <none> and password file <none>
I20260807 04:13:43.682533 3669 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:43.682672 3669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:43.689049 3669 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:13:43.692250 3685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:43.693756 3669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:13:43.694067 3669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "56908b36ebe2410abc5ff3e990d694e3"
format_stamp: "Formatted at 2026-08-07 04:13:20 on dist-test-slave-8n49"
server_key: "940e6b5623e5ae1eebcb16280052f255"
server_key_iv: "047c3be58ddf35a854a4c535f46eb504"
server_key_version: "encryptionkey@0"
I20260807 04:13:43.695185 3669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:43.709209 3669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:43.713650 3669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:43.713976 3669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:43.734632 3669 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:41531
I20260807 04:13:43.734653 3741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:41531 every 8 connection(s)
I20260807 04:13:43.736291 3669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:13:43.742767 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 3669
I20260807 04:13:43.743855 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:36177
--local_ip_for_outbound_sockets=127.18.193.1
--tserver_master_addrs=127.18.193.62:41531
--webserver_port=40387
--webserver_interface=127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:13:43.744593 3742 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:13:43.750289 3742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Bootstrap starting.
I20260807 04:13:43.773257 3742 log.cc:824] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:43.789319 3742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:13:43.790004 3742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Bootstrap complete.
I20260807 04:13:43.804163 3742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:43.805514 3742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56908b36ebe2410abc5ff3e990d694e3, State: Initialized, Role: FOLLOWER
I20260807 04:13:43.806350 3742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:43.806708 3742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:13:43.806849 3742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:13:43.807042 3742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:13:43.813360 3742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:43.814126 3742 leader_election.cc:302] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56908b36ebe2410abc5ff3e990d694e3; no voters:
I20260807 04:13:43.814942 3742 leader_election.cc:288] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:13:43.815078 3746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:13:43.816572 3746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 LEADER]: Becoming Leader. State: Replica: 56908b36ebe2410abc5ff3e990d694e3, State: Running, Role: LEADER
I20260807 04:13:43.817178 3746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:13:43.817425 3742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:13:43.819839 3748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56908b36ebe2410abc5ff3e990d694e3. Latest consensus state: current_term: 2 leader_uuid: "56908b36ebe2410abc5ff3e990d694e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } } }
I20260807 04:13:43.820230 3748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:13:43.820783 3747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "56908b36ebe2410abc5ff3e990d694e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } } }
I20260807 04:13:43.821107 3747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:13:43.831630 3753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:13:43.840642 3753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:13:43.842417 3753 catalog_manager.cc:1259] Loaded cluster ID: 19970674dbca4aeabef1db6157099964
I20260807 04:13:43.842581 3753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:13:43.846750 3753 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:13:43.848628 3753 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Loaded TSK: 0
I20260807 04:13:43.890743 3753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Generated new TSK 1
I20260807 04:13:43.891209 3753 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:13:44.143448 3744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:44.143822 3744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:44.144014 3744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:44.151688 3744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:44.151806 3744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:44.151981 3744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:13:44.162257 3744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:36177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=40387
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:44.163708 3744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:44.165198 3744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:44.180241 3770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:44.180241 3769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:44.181109 3772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:44.181663 3744 server_base.cc:1034] running on GCE node
I20260807 04:13:44.182365 3744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:44.183468 3744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:44.185217 3744 hybrid_clock.cc:648] HybridClock initialized: now 1786076024185144 us; error 56 us; skew 500 ppm
I20260807 04:13:44.185590 3744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:44 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:44.189610 3744 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:13:44.191344 3744 webserver.cc:460] Webserver started at http://127.18.193.1:40387/ using document root <none> and password file <none>
I20260807 04:13:44.192272 3744 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:44.192444 3744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:44.199743 3744 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:13:44.203189 3779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:44.204900 3744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:13:44.205116 3744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "9c63e16bfee648fe90400cdff5781fb1"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "3d51b549ecbddb93de819667ee4ddbc8"
server_key_iv: "daec79c24691f22575391899c6f5f5e0"
server_key_version: "encryptionkey@0"
I20260807 04:13:44.206604 3744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:44.226114 3744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:44.231455 3744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:44.231746 3744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:44.232944 3744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:44.234593 3744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:44.234710 3744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:44.234812 3744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:44.234885 3744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:44.270406 3744 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:36177
I20260807 04:13:44.270519 3901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:36177 every 8 connection(s)
I20260807 04:13:44.271891 3744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:13:44.276736 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 3744
I20260807 04:13:44.277375 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:45411
--local_ip_for_outbound_sockets=127.18.193.2
--tserver_master_addrs=127.18.193.62:41531
--webserver_port=35947
--webserver_interface=127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:13:44 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:44.295514 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:44.275016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:35103)
0807 04:13:44.275903 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:44.275947 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:44.277882 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:44.279198 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:44.279208 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:44.279839 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:13:44.280355 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:44.280372 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.282139 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.282145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:44.283348 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:44.283361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.284392 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.284400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:44.284717 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:44.285433 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:13:44.285456 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:13:44.287613 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:13:44.291093 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:44.291102 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:44.291116 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:44.291553 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:44.292045 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:44.292050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:44.292053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:44.292236 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:44.293043 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:13:44.293052 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:13:44.293374 (+ 322us) client_negotiation.cc:769] Sending connection context
0807 04:13:44.293663 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 04:13:44.294197 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":362,"threads_started":1}
I20260807 04:13:44.295499 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:44.274860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:35103 (local address 127.18.193.62:41531)
0807 04:13:44.275409 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:44.275428 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:44.276083 (+ 655us) server_negotiation.cc:407] Connection header received
0807 04:13:44.278390 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:44.278402 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:44.278705 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:44.279079 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:44.280642 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.281950 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.283501 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.284129 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.284356 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:44.287883 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:44.287891 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:44.287904 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:44.287938 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:44.290934 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:44.291714 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:44.291721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:44.291730 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:44.291918 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:44.292357 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:44.292363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:44.292380 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:44.292888 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:44.293102 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:44.293880 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 04:13:44.294268 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":226,"threads_started":1}
I20260807 04:13:44.298836 3902 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41531
I20260807 04:13:44.299274 3902 heartbeater.cc:464] Registering TS with master...
I20260807 04:13:44.300331 3902 heartbeater.cc:511] Master 127.18.193.62:41531 requested a full tablet report, sending...
I20260807 04:13:44.303133 3704 ts_manager.cc:194] Registered new tserver with Master: 9c63e16bfee648fe90400cdff5781fb1 (127.18.193.1:36177)
I20260807 04:13:44.305578 3704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:35103
W20260807 04:13:44.620909 3907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:44.621201 3907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:44.621299 3907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:44.628626 3907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:44.628743 3907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:44.628873 3907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:13:44.639201 3907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:45411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=35947
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:44.640475 3907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:44.641855 3907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:44.655442 3914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:44.655541 3913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:44.655565 3918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:44.656281 3907 server_base.cc:1034] running on GCE node
I20260807 04:13:44.656947 3907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:44.657851 3907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:44.659070 3907 hybrid_clock.cc:648] HybridClock initialized: now 1786076024659023 us; error 39 us; skew 500 ppm
I20260807 04:13:44.659428 3907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:44 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:44.662684 3907 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:13:44.664119 3907 webserver.cc:460] Webserver started at http://127.18.193.2:35947/ using document root <none> and password file <none>
I20260807 04:13:44.664880 3907 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:44.665048 3907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:44.670352 3907 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 04:13:44.673166 3925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:44.674582 3907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:13:44.674772 3907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "2412993129d64d75ad7bc8f852bbc0e5"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "192450a0d597abaf756f1aaf34a34d81"
server_key_iv: "4780b4b7ed43eb5bf1a409c629a68b07"
server_key_version: "encryptionkey@0"
I20260807 04:13:44.675762 3907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:44.689585 3907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:44.693858 3907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:44.694216 3907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:44.695192 3907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:44.696539 3907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:44.696715 3907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:44.696857 3907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:44.696956 3907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:44.731357 3907 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:45411
I20260807 04:13:44.731424 4044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:45411 every 8 connection(s)
I20260807 04:13:44.733054 3907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:13:44.735347 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 3907
I20260807 04:13:44.736117 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:32777
--local_ip_for_outbound_sockets=127.18.193.3
--tserver_master_addrs=127.18.193.62:41531
--webserver_port=42517
--webserver_interface=127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:13:44 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:44.758180 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:44.735920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:60643 (local address 127.18.193.62:41531)
0807 04:13:44.736159 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:44.736166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:44.738779 (+ 2613us) server_negotiation.cc:407] Connection header received
0807 04:13:44.742931 (+ 4152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:44.742940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:44.743032 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:44.743098 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:44.745227 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.746278 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.748077 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.748654 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.748786 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:44.752304 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:44.752309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:44.752312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:44.752344 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:44.755066 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:44.755793 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:44.755800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:44.755803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:44.755929 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:44.756361 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:44.756366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:44.756370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:44.756734 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:44.756890 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:44.757583 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 04:13:44.757811 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 04:13:44.759204 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:44.736395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:60643)
0807 04:13:44.738612 (+ 2217us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:44.738663 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:44.742634 (+ 3971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:44.743585 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:44.743598 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:44.744290 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:13:44.745044 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:44.745067 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.746485 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.746492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:44.747921 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:44.747933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:44.748891 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:44.748900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:44.749136 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:44.749882 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:13:44.749904 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:13:44.752066 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:13:44.755213 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:44.755221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:44.755237 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:44.755670 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:44.756047 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:44.756052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:44.756056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:44.756240 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:44.756864 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:13:44.756872 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:13:44.757210 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 04:13:44.757476 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 04:13:44.758007 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1935,"thread_start_us":1727,"threads_started":1}
I20260807 04:13:44.760885 4045 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41531
I20260807 04:13:44.761214 4045 heartbeater.cc:464] Registering TS with master...
I20260807 04:13:44.761983 4045 heartbeater.cc:511] Master 127.18.193.62:41531 requested a full tablet report, sending...
I20260807 04:13:44.763492 3704 ts_manager.cc:194] Registered new tserver with Master: 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411)
I20260807 04:13:44.764356 3704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:60643
W20260807 04:13:45.082000 4048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:13:45.082274 4048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:13:45.082430 4048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:13:45.089803 4048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:13:45.089912 4048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:13:45.090137 4048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:13:45.099944 4048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:32777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=42517
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:13:45.101297 4048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:13:45.102602 4048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:13:45.115717 4054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:45.115799 4057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:13:45.115799 4055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:13:45.116489 4048 server_base.cc:1034] running on GCE node
I20260807 04:13:45.117075 4048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:13:45.118012 4048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:13:45.119218 4048 hybrid_clock.cc:648] HybridClock initialized: now 1786076025119156 us; error 49 us; skew 500 ppm
I20260807 04:13:45.119498 4048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:13:45 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076025, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:13:45.122578 4048 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:13:45.123972 4048 webserver.cc:460] Webserver started at http://127.18.193.3:42517/ using document root <none> and password file <none>
I20260807 04:13:45.124701 4048 fs_manager.cc:359] Metadata directory not provided
I20260807 04:13:45.124855 4048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:13:45.130300 4048 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:13:45.133179 4064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:45.134577 4048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:13:45.134752 4048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "9db9764d15a0471f9f2830e8c45237c3"
format_stamp: "Formatted at 2026-08-07 04:13:22 on dist-test-slave-8n49"
server_key: "a3569a85d330a6b99a7c98ad0b0b1545"
server_key_iv: "aac77176d9961b24eb7dd895845cbef4"
server_key_version: "encryptionkey@0"
I20260807 04:13:45.136188 4048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:13:45.188740 4048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:13:45.192773 4048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:13:45.193053 4048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:13:45.194350 4048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:13:45.195737 4048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:13:45.195873 4048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:45.195971 4048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:13:45.196030 4048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:13:45.230158 4048 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:32777
I20260807 04:13:45.230255 4183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:32777 every 8 connection(s)
I20260807 04:13:45.231617 4048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:13:45.239768 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 4048
Aug 07 04:13:45 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076025, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:13:45.252908 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.234301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:38527 (local address 127.18.193.62:41531)
0807 04:13:45.234486 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.234491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.235606 (+ 1115us) server_negotiation.cc:407] Connection header received
0807 04:13:45.237388 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.237393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.237471 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.237559 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:45.239060 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.240026 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.242066 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.242591 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.242739 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.246319 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:45.246323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:45.246327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:45.246357 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:45.249411 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.250191 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.250197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.250200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.250349 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.250832 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.250836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.250838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.251128 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:45.251246 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.252195 (+ 949us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.252465 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:13:45.254112 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.234731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:38527)
0807 04:13:45.235437 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.235477 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:45.237129 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:45.237770 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:45.237778 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:45.238362 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:13:45.238898 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.238915 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.240173 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.240180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.241927 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.241939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.242836 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.242845 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.243087 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.243766 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:13:45.243792 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:13:45.246102 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:13:45.249550 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:45.249559 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:45.249575 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:45.250033 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:45.250474 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:13:45.250477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:13:45.250479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:13:45.250705 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:13:45.251269 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:13:45.251280 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:13:45.251753 (+ 473us) client_negotiation.cc:769] Sending connection context
0807 04:13:45.252062 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 04:13:45.252651 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":288,"threads_started":1}
I20260807 04:13:45.255676 4184 heartbeater.cc:347] Connected to a master server at 127.18.193.62:41531
I20260807 04:13:45.256017 4184 heartbeater.cc:464] Registering TS with master...
I20260807 04:13:45.256901 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.240863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.18.193.62:41531)
0807 04:13:45.241466 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.241475 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.242475 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 04:13:45.242681 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.242686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.242753 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.242873 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:45.244454 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.245582 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.246956 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.247665 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.247810 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.250727 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:45.250735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:45.250743 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:45.250785 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:45.254012 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.254803 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.254807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.254809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.254966 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.255496 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.255501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.255504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.255775 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:45.256022 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.256404 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.256643 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":286,"threads_started":1}
I20260807 04:13:45.256958 4184 heartbeater.cc:511] Master 127.18.193.62:41531 requested a full tablet report, sending...
I20260807 04:13:45.258517 3704 ts_manager.cc:194] Registered new tserver with Master: 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777)
I20260807 04:13:45.259475 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:13:45.259562 3704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:38527
I20260807 04:13:45.266237 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.260627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35502 (local address 127.18.193.62:41531)
0807 04:13:45.260804 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.260812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.261168 (+ 356us) server_negotiation.cc:407] Connection header received
0807 04:13:45.261408 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.261412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.261536 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.261678 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:13:45.262118 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.262797 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.264225 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.264705 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.264838 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.265330 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:13:45.265584 (+ 254us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 04:13:45.265971 (+ 387us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35502: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 04:13:45.266336 4187 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35502: token expired
W20260807 04:13:45.266928 3620 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.193.62:41531)
I20260807 04:13:45.281262 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.267450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35508 (local address 127.18.193.62:41531)
0807 04:13:45.267624 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.267628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.267644 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:13:45.267734 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.267737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.267786 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.267859 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:45.269223 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.269999 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.271950 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.272323 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.272431 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.275745 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:45.275750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:45.275753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:45.275782 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:45.278560 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.279238 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.279242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.279244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.279365 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.279717 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.279723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.279726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.280106 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:45.280370 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.280699 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.280922 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:13:45.284502 3704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:13:45.291889 3704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:13:45.309306 3902 heartbeater.cc:503] Master 127.18.193.62:41531 was elected leader, sending a full tablet report...
I20260807 04:13:45.315658 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.303618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51158)
0807 04:13:45.305679 (+ 2061us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.305701 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:45.305825 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:45.307484 (+ 1659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:45.307495 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:45.307533 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:45.308036 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.308053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.310340 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.310345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.312888 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.312903 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.314781 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.314790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.315022 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.315083 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 04:13:45.315190 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:13:45.315373 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"thread_start_us":2326,"threads_started":1}
I20260807 04:13:45.316903 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.305625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:32777 (local address 127.0.0.1:60872)
0807 04:13:45.306309 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.306328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:45.307712 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:45.308943 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:45.308947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:45.308973 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:45.309403 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.309419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.312049 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.312055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.314196 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.314216 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.316091 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.316098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.316263 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.316311 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 04:13:45.316373 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 04:13:45.316504 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":788,"threads_started":1}
I20260807 04:13:45.317037 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.304276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.18.193.1:36177)
0807 04:13:45.306701 (+ 2425us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.306713 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.306749 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:13:45.306868 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.306878 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.307198 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.308260 (+ 1062us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:45.308544 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.310186 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.313048 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.314533 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.316179 (+ 1646us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.316443 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.316511 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.316698 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2221,"thread_start_us":2084,"threads_started":1}
I20260807 04:13:45.317309 4194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.303433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45411 (local address 127.0.0.1:43084)
0807 04:13:45.305095 (+ 1662us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.305141 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:45.305333 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:45.309256 (+ 3923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:45.309261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:45.309290 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:45.309661 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.309673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.313087 (+ 3414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.313094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.314562 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.314577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.316470 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.316477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.316684 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.316716 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:13:45.316809 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:13:45.316938 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1363,"thread_start_us":1173,"threads_started":1}
I20260807 04:13:45.317480 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.306219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.18.193.3:32777)
0807 04:13:45.308438 (+ 2219us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.308446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.308478 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:13:45.308602 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.308611 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.308765 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.308967 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:45.310087 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.311907 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.314382 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.315830 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.316682 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.316891 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.316966 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.317128 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2015,"thread_start_us":1836,"threads_started":1}
I20260807 04:13:45.317999 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.303622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.18.193.2:45411)
0807 04:13:45.308711 (+ 5089us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.308718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.308744 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:13:45.308835 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.308842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.309049 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.311074 (+ 2025us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:45.311296 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.312936 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.314772 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.316223 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.316365 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.316546 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.317531 (+ 985us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.317690 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4907,"thread_start_us":4756,"threads_started":1}
I20260807 04:13:45.320995 3832 tablet_service.cc:1467] Processing CreateTablet for tablet b7488f18b72d447b8aeed6f8ec69f324 (DEFAULT_TABLE table=test-table [id=c24f1ca552ba40f2b839fed4662b751e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:45.321056 4115 tablet_service.cc:1467] Processing CreateTablet for tablet b7488f18b72d447b8aeed6f8ec69f324 (DEFAULT_TABLE table=test-table [id=c24f1ca552ba40f2b839fed4662b751e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:45.321216 3976 tablet_service.cc:1467] Processing CreateTablet for tablet b7488f18b72d447b8aeed6f8ec69f324 (DEFAULT_TABLE table=test-table [id=c24f1ca552ba40f2b839fed4662b751e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:13:45.322273 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7488f18b72d447b8aeed6f8ec69f324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:45.322398 4115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7488f18b72d447b8aeed6f8ec69f324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:45.322400 3976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7488f18b72d447b8aeed6f8ec69f324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:13:45.334426 4200 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Bootstrap starting.
I20260807 04:13:45.337525 4201 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Bootstrap starting.
I20260807 04:13:45.338375 4200 tablet_bootstrap.cc:654] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:45.338366 4202 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Bootstrap starting.
I20260807 04:13:45.339728 4200 log.cc:824] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:45.342197 4201 tablet_bootstrap.cc:654] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:45.343432 4201 log.cc:824] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:45.343860 4202 tablet_bootstrap.cc:654] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Neither blocks nor log segments found. Creating new log.
I20260807 04:13:45.345095 4202 log.cc:824] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Log is configured to *not* fsync() on all Append() calls
I20260807 04:13:45.346141 4200 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: No bootstrap required, opened a new log
I20260807 04:13:45.346216 4201 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: No bootstrap required, opened a new log
I20260807 04:13:45.346437 4200 ts_tablet_manager.cc:1397] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260807 04:13:45.346511 4201 ts_tablet_manager.cc:1397] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260807 04:13:45.348099 4202 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: No bootstrap required, opened a new log
I20260807 04:13:45.348344 4202 ts_tablet_manager.cc:1397] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 04:13:45.354161 4201 raft_consensus.cc:348] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.354552 4201 raft_consensus.cc:374] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:45.354681 4201 raft_consensus.cc:729] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2412993129d64d75ad7bc8f852bbc0e5, State: Initialized, Role: FOLLOWER
I20260807 04:13:45.355306 4201 consensus_queue.cc:260] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.356436 4045 heartbeater.cc:503] Master 127.18.193.62:41531 was elected leader, sending a full tablet report...
I20260807 04:13:45.357000 4201 ts_tablet_manager.cc:1428] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260807 04:13:45.358632 4202 raft_consensus.cc:348] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.358635 4200 raft_consensus.cc:348] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.359056 4200 raft_consensus.cc:374] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:45.359061 4202 raft_consensus.cc:374] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:13:45.359161 4200 raft_consensus.cc:729] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c63e16bfee648fe90400cdff5781fb1, State: Initialized, Role: FOLLOWER
I20260807 04:13:45.359161 4202 raft_consensus.cc:729] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db9764d15a0471f9f2830e8c45237c3, State: Initialized, Role: FOLLOWER
I20260807 04:13:45.359757 4202 consensus_queue.cc:260] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.359758 4200 consensus_queue.cc:260] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.361362 4184 heartbeater.cc:503] Master 127.18.193.62:41531 was elected leader, sending a full tablet report...
I20260807 04:13:45.361994 4202 ts_tablet_manager.cc:1428] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260807 04:13:45.362325 4200 ts_tablet_manager.cc:1428] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260807 04:13:45.483296 4185 tablet.cc:2369] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:13:45.485164 4046 tablet.cc:2369] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:13:45.524413 3903 tablet.cc:2369] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:13:45.586678 4208 raft_consensus.cc:482] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:13:45.586969 4208 raft_consensus.cc:504] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.588742 4208 leader_election.cc:288] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411), 9c63e16bfee648fe90400cdff5781fb1 (127.18.193.1:36177)
I20260807 04:13:45.594321 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.589351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:32845 (local address 127.18.193.1:36177)
0807 04:13:45.589586 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.589592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.589614 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:13:45.589698 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.589711 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.589769 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.589871 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:45.590285 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.591041 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.592669 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.593455 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.593631 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.593708 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.593954 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.594075 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:13:45.594374 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.589122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.18.193.3:32845)
0807 04:13:45.589410 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.589434 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:45.589565 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:45.589898 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:45.589903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:45.589937 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:45.590145 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.590156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.591193 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.591198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.592491 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.592505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.593617 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.593623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.593801 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.593824 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:13:45.593929 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 04:13:45.594066 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":2}
I20260807 04:13:45.595199 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.589173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:35097 (local address 127.18.193.2:45411)
0807 04:13:45.589375 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.589382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.590159 (+ 777us) server_negotiation.cc:407] Connection header received
0807 04:13:45.590332 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.590336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.590393 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.590489 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:13:45.590974 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.591663 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.593443 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.594228 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.594402 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.594474 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.594812 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.594945 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:13:45.595547 3853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7488f18b72d447b8aeed6f8ec69f324" candidate_uuid: "9db9764d15a0471f9f2830e8c45237c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c63e16bfee648fe90400cdff5781fb1" is_pre_election: true
I20260807 04:13:45.595862 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.589133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45411 (local address 127.18.193.3:35097)
0807 04:13:45.590053 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.590067 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:13:45.590178 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:13:45.590504 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:13:45.590508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:13:45.590530 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:13:45.590832 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.590840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.591798 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.591803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.593290 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:13:45.593301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.594436 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.594442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:13:45.594644 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.594672 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 04:13:45.595386 (+ 714us) client_negotiation.cc:240] Negotiation successful
0807 04:13:45.595523 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":87,"thread_start_us":486,"threads_started":1}
I20260807 04:13:45.596144 3853 raft_consensus.cc:2454] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db9764d15a0471f9f2830e8c45237c3 in term 0.
I20260807 04:13:45.596909 4065 leader_election.cc:302] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c63e16bfee648fe90400cdff5781fb1, 9db9764d15a0471f9f2830e8c45237c3; no voters:
I20260807 04:13:45.596896 3996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7488f18b72d447b8aeed6f8ec69f324" candidate_uuid: "9db9764d15a0471f9f2830e8c45237c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2412993129d64d75ad7bc8f852bbc0e5" is_pre_election: true
I20260807 04:13:45.597288 4208 raft_consensus.cc:2781] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:13:45.597451 4208 raft_consensus.cc:482] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:13:45.597401 3996 raft_consensus.cc:2454] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db9764d15a0471f9f2830e8c45237c3 in term 0.
I20260807 04:13:45.597576 4208 raft_consensus.cc:3037] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:45.599893 4208 raft_consensus.cc:504] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.600610 4208 leader_election.cc:288] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [CANDIDATE]: Term 1 election: Requested vote from peers 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411), 9c63e16bfee648fe90400cdff5781fb1 (127.18.193.1:36177)
I20260807 04:13:45.600936 3996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7488f18b72d447b8aeed6f8ec69f324" candidate_uuid: "9db9764d15a0471f9f2830e8c45237c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2412993129d64d75ad7bc8f852bbc0e5"
I20260807 04:13:45.601140 3996 raft_consensus.cc:3037] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:45.601121 3853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7488f18b72d447b8aeed6f8ec69f324" candidate_uuid: "9db9764d15a0471f9f2830e8c45237c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c63e16bfee648fe90400cdff5781fb1"
I20260807 04:13:45.601325 3853 raft_consensus.cc:3037] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:13:45.603354 3996 raft_consensus.cc:2454] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db9764d15a0471f9f2830e8c45237c3 in term 1.
I20260807 04:13:45.603524 3853 raft_consensus.cc:2454] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db9764d15a0471f9f2830e8c45237c3 in term 1.
I20260807 04:13:45.603762 4066 leader_election.cc:302] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2412993129d64d75ad7bc8f852bbc0e5, 9db9764d15a0471f9f2830e8c45237c3; no voters:
I20260807 04:13:45.604110 4208 raft_consensus.cc:2781] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:13:45.604620 4208 raft_consensus.cc:686] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9db9764d15a0471f9f2830e8c45237c3, State: Running, Role: LEADER
I20260807 04:13:45.605077 4208 consensus_queue.cc:237] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:13:45.609405 3703 catalog_manager.cc:5495] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3). New cstate: current_term: 1 leader_uuid: "9db9764d15a0471f9f2830e8c45237c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } health_report { overall_health: HEALTHY } } }
I20260807 04:13:45.680269 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.674610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.18.193.3:32777)
0807 04:13:45.674785 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.674789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.674885 (+ 96us) server_negotiation.cc:407] Connection header received
0807 04:13:45.675093 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.675096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.675177 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.675311 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:13:45.675732 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.676450 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.678036 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.678568 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.678667 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.679211 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:13:45.679704 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:13:45.679821 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.679925 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.679992 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:13:45.688508 3853 raft_consensus.cc:1261] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Refusing update from remote peer 9db9764d15a0471f9f2830e8c45237c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:13:45.688498 3996 raft_consensus.cc:1261] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Refusing update from remote peer 9db9764d15a0471f9f2830e8c45237c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:13:45.689510 4210 consensus_queue.cc:1035] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:13:45.689929 4208 consensus_queue.cc:1035] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:13:45.699561 4213 mvcc.cc:205] Tried to move back new op lower bound from 7315767401209520128 to 7315767400881917952. Current Snapshot: MvccSnapshot[applied={T|T < 7315767401209520128}]
I20260807 04:13:45.700598 4215 mvcc.cc:205] Tried to move back new op lower bound from 7315767401209520128 to 7315767400881917952. Current Snapshot: MvccSnapshot[applied={T|T < 7315767401209520128}]
I20260807 04:13:45.714023 3702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:13:45.714463 3702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:13:45.718988 3702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Sending DeleteTablet for 3 replicas of tablet b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:13:45.720146 3976 tablet_service.cc:1514] Processing DeleteTablet for tablet b7488f18b72d447b8aeed6f8ec69f324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:45 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:43084
I20260807 04:13:45.720530 3832 tablet_service.cc:1514] Processing DeleteTablet for tablet b7488f18b72d447b8aeed6f8ec69f324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:45 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:51158
I20260807 04:13:45.721330 4115 tablet_service.cc:1514] Processing DeleteTablet for tablet b7488f18b72d447b8aeed6f8ec69f324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:13:45 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:60872
I20260807 04:13:45.722227 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 3744
I20260807 04:13:45.735898 4222 tablet_replica.cc:330] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: stopping tablet replica
I20260807 04:13:45.736529 4222 raft_consensus.cc:2229] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:13:45.737154 4222 raft_consensus.cc:2258] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:45.739524 4222 ts_tablet_manager.cc:1905] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:45.746695 4222 ts_tablet_manager.cc:1918] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:13:45.746855 4222 log.cc:1196] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:13:45.747390 4222 ts_tablet_manager.cc:1939] T b7488f18b72d447b8aeed6f8ec69f324 P 2412993129d64d75ad7bc8f852bbc0e5: Deleting consensus metadata
I20260807 04:13:45.749274 3687 catalog_manager.cc:4841] TS 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411): tablet b7488f18b72d447b8aeed6f8ec69f324 (table test-table [id=c24f1ca552ba40f2b839fed4662b751e]) successfully deleted
I20260807 04:13:45.752247 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.723660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.18.193.1:36177)
0807 04:13:45.723826 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.723831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.723849 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:13:45.723905 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.723911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.723995 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.724095 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:45.726332 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.727595 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.729608 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.730342 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.730486 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.741657 (+ 11171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:45.741679 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:45.741696 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:45.741743 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:45.746144 (+ 4401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.748223 (+ 2079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.748234 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.748238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.748420 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.750130 (+ 1710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.750137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.750143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.750697 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:45.751029 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.751369 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.751693 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:13:45.753084 3897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:45.758684 4224 tablet_replica.cc:330] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: stopping tablet replica
I20260807 04:13:45.759198 4224 raft_consensus.cc:2229] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:13:45.759898 4224 raft_consensus.cc:2258] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:45.762049 4224 ts_tablet_manager.cc:1905] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:13:45.769304 4224 ts_tablet_manager.cc:1918] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:13:45.769565 4224 log.cc:1196] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:13:45.770249 4224 ts_tablet_manager.cc:1939] T b7488f18b72d447b8aeed6f8ec69f324 P 9db9764d15a0471f9f2830e8c45237c3: Deleting consensus metadata
I20260807 04:13:45.772141 3689 catalog_manager.cc:4841] TS 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777): tablet b7488f18b72d447b8aeed6f8ec69f324 (table test-table [id=c24f1ca552ba40f2b839fed4662b751e]) successfully deleted
I20260807 04:13:45.889736 4223 tablet_replica.cc:330] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: stopping tablet replica
I20260807 04:13:45.890240 4223 raft_consensus.cc:2229] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:13:45.890784 4223 raft_consensus.cc:2258] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:13:45.890794 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 3744
I20260807 04:13:45.910236 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 3907
W20260807 04:13:45.910379 3686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.193.1:36177 (error 108)
I20260807 04:13:45.913396 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.912423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51186)
0807 04:13:45.912986 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.913135 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":294,"threads_started":1}
W20260807 04:13:45.913712 3686 catalog_manager.cc:4584] TS 9c63e16bfee648fe90400cdff5781fb1 (127.18.193.1:36177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7488f18b72d447b8aeed6f8ec69f324: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
I20260807 04:13:45.926232 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.911614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.18.193.2:45411)
0807 04:13:45.911777 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:45.911781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:45.911797 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:13:45.911852 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:45.911856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:45.911920 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:45.912037 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:45.913840 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.914628 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.916098 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:45.916509 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:45.916708 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:45.919580 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:45.919599 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:45.919619 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:45.919656 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:45.922927 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.923783 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.923800 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.923803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.923957 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:45.924404 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:45.924411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:45.924415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:45.924902 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:45.925231 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:45.925601 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 04:13:45.925847 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 04:13:45.927068 4039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:45.967495 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.966921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51202)
0807 04:13:45.967191 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.967320 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 04:13:45.968724 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:45.968277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51216)
0807 04:13:45.968453 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:45.968540 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 04:13:46.019052 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 3907
I20260807 04:13:46.029736 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:46.029156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51228)
0807 04:13:46.029422 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:46.029544 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 04:13:46.031096 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:46.030605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51232)
0807 04:13:46.030801 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:46.030901 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 04:13:46.039870 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 4048
I20260807 04:13:46.054796 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:46.040837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.18.193.3:32777)
0807 04:13:46.041101 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 04:13:46.041107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:13:46.041124 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:13:46.041196 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:13:46.041200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:13:46.041262 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:13:46.041390 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:13:46.042715 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:46.043528 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:46.044983 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:13:46.045441 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:13:46.045592 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:13:46.048308 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:13:46.048324 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:13:46.048339 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:13:46.048378 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:13:46.051690 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:46.052489 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:46.052504 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:46.052506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:46.052724 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:13:46.053119 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:13:46.053126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:13:46.053129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:13:46.053485 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:13:46.053829 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 04:13:46.054158 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 04:13:46.054395 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 04:13:46.055526 4179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:46.111987 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:46.111443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51240)
0807 04:13:46.111721 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:46.111817 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 04:13:46.113164 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:13:46.112761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:36177 (local address 127.0.0.1:51248)
0807 04:13:46.112955 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 04:13:46.113023 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:36177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 04:13:46.178496 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 4048
I20260807 04:13:46.200152 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 3669
I20260807 04:13:46.201350 3737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:13:46.317019 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 3669
I20260807 04:14:06.341879 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41531
--webserver_interface=127.18.193.62
--webserver_port=39825
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:41531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:06.685047 4232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:06.685339 4232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:06.685431 4232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:06.692775 4232 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 04:14:06.692878 4232 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 04:14:06.693095 4232 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 04:14:06.693176 4232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:06.693240 4232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:06.693312 4232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:06.693359 4232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:06.693424 4232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:06.703681 4232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:41531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:41531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=39825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:06.704975 4232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:06.706269 4232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:06.718297 4237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:06.718266 4238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:06.718326 4240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:06.719033 4232 server_base.cc:1034] running on GCE node
I20260807 04:14:06.719844 4232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:06.721349 4232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:06.722595 4232 hybrid_clock.cc:648] HybridClock initialized: now 1786076046722518 us; error 61 us; skew 500 ppm
I20260807 04:14:06.722954 4232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:06 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076046, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:06.730926 4232 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:14:06.732645 4232 webserver.cc:460] Webserver started at http://127.18.193.62:39825/ using document root <none> and password file <none>
I20260807 04:14:06.733431 4232 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:06.733578 4232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:06.739367 4232 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:14:06.742268 4248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:06.743535 4232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:14:06.743723 4232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "56908b36ebe2410abc5ff3e990d694e3"
format_stamp: "Formatted at 2026-08-07 04:13:20 on dist-test-slave-8n49"
server_key: "940e6b5623e5ae1eebcb16280052f255"
server_key_iv: "047c3be58ddf35a854a4c535f46eb504"
server_key_version: "encryptionkey@0"
I20260807 04:14:06.744784 4232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:06.762199 4232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:06.766824 4232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:06.767150 4232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:06.787322 4232 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:41531
I20260807 04:14:06.787352 4300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:41531 every 8 connection(s)
I20260807 04:14:06.789270 4232 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:06.791879 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 4232
I20260807 04:14:06.792855 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:36177
--local_ip_for_outbound_sockets=127.18.193.1
--tserver_master_addrs=127.18.193.62:41531
--webserver_port=40387
--webserver_interface=127.18.193.1
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:14:06.796782 4302 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:14:06.800007 4302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Bootstrap starting.
I20260807 04:14:06.820453 4302 log.cc:824] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:06.836272 4302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:14:06.836891 4302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Bootstrap complete.
I20260807 04:14:06.846544 4302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:14:06.847397 4302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56908b36ebe2410abc5ff3e990d694e3, State: Initialized, Role: FOLLOWER
I20260807 04:14:06.848142 4302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:14:06.848379 4302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:06.848522 4302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:06.848742 4302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 04:14:06.852752 4302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:14:06.853235 4302 leader_election.cc:302] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56908b36ebe2410abc5ff3e990d694e3; no voters:
I20260807 04:14:06.853928 4302 leader_election.cc:288] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 04:14:06.854071 4306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 04:14:06.855276 4306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [term 3 LEADER]: Becoming Leader. State: Replica: 56908b36ebe2410abc5ff3e990d694e3, State: Running, Role: LEADER
I20260807 04:14:06.855893 4306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } }
I20260807 04:14:06.856127 4302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:06.858495 4308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56908b36ebe2410abc5ff3e990d694e3. Latest consensus state: current_term: 3 leader_uuid: "56908b36ebe2410abc5ff3e990d694e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } } }
I20260807 04:14:06.859025 4308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:06.859117 4307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "56908b36ebe2410abc5ff3e990d694e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56908b36ebe2410abc5ff3e990d694e3" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 41531 } } }
I20260807 04:14:06.859532 4307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:06.862808 4314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:06.871011 4314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:06.872543 4314 catalog_manager.cc:1259] Loaded cluster ID: 19970674dbca4aeabef1db6157099964
I20260807 04:14:06.872701 4314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:06.876732 4314 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:06.878376 4314 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Loaded TSK: 0
I20260807 04:14:06.878551 4314 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56908b36ebe2410abc5ff3e990d694e3: Loaded TSK: 1
F20260807 04:14:06.879707 4314 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6285ac99ad google::LogMessage::Fail() at ??:0
@ 0x7f6285acc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6285ac94de google::LogMessage::Flush() at ??:0
@ 0x7f6285acaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f628982d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f628981c73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f629b4fc1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f629b4fb783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f629b5872a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f629b5870c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f629b5d15e9 std::function<>::operator()() at ??:0
@ 0x7f629b4ffcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f629b4fdc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f629b585610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f629b585439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6288715345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f628871891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6288718799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f62886f3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f628b9cd6db start_thread at ??:0
@ 0x7f628496361f clone at ??:0
W20260807 04:14:07.207693 4304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:07.208058 4304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:07.208236 4304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:07.215989 4304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:07.216145 4304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:07.216318 4304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:14:07.226505 4304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:36177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=40387
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:07.227854 4304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:07.229341 4304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:07.243412 4352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:07.243477 4354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:07.243539 4351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:07.244467 4304 server_base.cc:1034] running on GCE node
I20260807 04:14:07.245128 4304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:07.246040 4304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:07.247258 4304 hybrid_clock.cc:648] HybridClock initialized: now 1786076047247192 us; error 57 us; skew 500 ppm
I20260807 04:14:07.247612 4304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:07 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076047, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:07.250929 4304 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:14:07.252707 4304 webserver.cc:460] Webserver started at http://127.18.193.1:40387/ using document root <none> and password file <none>
I20260807 04:14:07.253469 4304 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:07.253613 4304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:07.259094 4304 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 04:14:07.262029 4361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:07.263550 4304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:07.263752 4304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "9c63e16bfee648fe90400cdff5781fb1"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "3d51b549ecbddb93de819667ee4ddbc8"
server_key_iv: "daec79c24691f22575391899c6f5f5e0"
server_key_version: "encryptionkey@0"
I20260807 04:14:07.264935 4304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:07.297022 4304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:07.301369 4304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:07.301689 4304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:07.302845 4304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:07.304986 4369 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 04:14:07.310580 4304 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 04:14:07.310750 4304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260807 04:14:07.310927 4304 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 04:14:07.314519 4304 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 04:14:07.314644 4304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.002s
I20260807 04:14:07.315012 4369 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Bootstrap starting.
I20260807 04:14:07.342720 4369 log.cc:824] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:07.358244 4369 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:14:07.358829 4369 tablet_bootstrap.cc:492] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Bootstrap complete.
I20260807 04:14:07.360432 4369 ts_tablet_manager.cc:1397] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Time spent bootstrapping tablet: real 0.046s user 0.025s sys 0.015s
I20260807 04:14:07.360613 4304 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:36177
I20260807 04:14:07.360679 4482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:36177 every 8 connection(s)
I20260807 04:14:07.362008 4304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:14:07.363446 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 4304
I20260807 04:14:07.364264 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:45411
--local_ip_for_outbound_sockets=127.18.193.2
--tserver_master_addrs=127.18.193.62:41531
--webserver_port=35947
--webserver_interface=127.18.193.2
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:14:07.373162 4369 raft_consensus.cc:348] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:14:07.375125 4369 raft_consensus.cc:729] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c63e16bfee648fe90400cdff5781fb1, State: Initialized, Role: FOLLOWER
I20260807 04:14:07.376278 4369 consensus_queue.cc:260] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:14:07.378988 4369 ts_tablet_manager.cc:1428] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1: Time spent starting tablet: real 0.018s user 0.012s sys 0.002s
I20260807 04:14:07.382105 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:07.372004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:52743)
0807 04:14:07.380507 (+ 8503us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:07.380893 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8156,"thread_start_us":7854,"threads_started":1}
W20260807 04:14:07.382810 4483 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:41531 (0 consecutive failures): Network error: Failed to ping master at 127.18.193.62:41531: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
I20260807 04:14:07.383809 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:07.383462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:57091)
0807 04:14:07.383626 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:07.383701 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 04:14:07.384783 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:07.384327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:49195)
0807 04:14:07.384514 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:07.384659 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 04:14:07.385082 4483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:14:07.711288 4486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:07.711563 4486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:07.711710 4486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:07.719038 4486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:07.719152 4486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:07.719331 4486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:14:07.729193 4486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:45411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=35947
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:07.730540 4486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:07.731878 4486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:07.745744 4492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:07.745658 4495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:07.745689 4493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:07.746765 4486 server_base.cc:1034] running on GCE node
I20260807 04:14:07.747330 4486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:07.748340 4486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:07.749547 4486 hybrid_clock.cc:648] HybridClock initialized: now 1786076047749481 us; error 60 us; skew 500 ppm
I20260807 04:14:07.749868 4486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:07 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076047, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:07.753053 4486 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:14:07.754500 4486 webserver.cc:460] Webserver started at http://127.18.193.2:35947/ using document root <none> and password file <none>
I20260807 04:14:07.755136 4486 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:07.755237 4486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:07.760866 4486 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:07.763643 4503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:07.765056 4486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:07.765240 4486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "2412993129d64d75ad7bc8f852bbc0e5"
format_stamp: "Formatted at 2026-08-07 04:13:21 on dist-test-slave-8n49"
server_key: "192450a0d597abaf756f1aaf34a34d81"
server_key_iv: "4780b4b7ed43eb5bf1a409c629a68b07"
server_key_version: "encryptionkey@0"
I20260807 04:14:07.766251 4486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:07.805684 4486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:07.810290 4486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:07.810523 4486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:07.811504 4486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:07.813035 4486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:07.813146 4486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:07.813238 4486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:07.813301 4486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:07.851730 4486 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:45411
I20260807 04:14:07.851843 4625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:45411 every 8 connection(s)
I20260807 04:14:07.853474 4486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:14:07.858182 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 4486
I20260807 04:14:07.858578 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:07.856631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:35513)
0807 04:14:07.857387 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:07.857632 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":282,"threads_started":1}
I20260807 04:14:07.858966 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:32777
--local_ip_for_outbound_sockets=127.18.193.3
--tserver_master_addrs=127.18.193.62:41531
--webserver_port=42517
--webserver_interface=127.18.193.3
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:07.859277 4626 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:41531 (0 consecutive failures): Network error: Failed to ping master at 127.18.193.62:41531: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
I20260807 04:14:07.860154 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:07.859801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:57105)
0807 04:14:07.859962 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:07.860044 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 04:14:07.861047 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:07.860719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:49037)
0807 04:14:07.860858 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:07.860938 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:14:07.861275 4626 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:14:08.211181 4629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:08.211479 4629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:08.211573 4629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:08.218956 4629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:08.219066 4629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:08.219180 4629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:14:08.228924 4629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:32777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=42517
--tserver_master_addrs=127.18.193.62:41531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:08.230168 4629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:08.231438 4629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:08.244832 4634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:08.244791 4637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:08.244874 4635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:08.245985 4629 server_base.cc:1034] running on GCE node
I20260807 04:14:08.246565 4629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:08.247457 4629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:08.248659 4629 hybrid_clock.cc:648] HybridClock initialized: now 1786076048248614 us; error 36 us; skew 500 ppm
I20260807 04:14:08.248991 4629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:08 dist-test-slave-8n49 krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076048, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:08.253036 4629 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:14:08.254546 4629 webserver.cc:460] Webserver started at http://127.18.193.3:42517/ using document root <none> and password file <none>
I20260807 04:14:08.255235 4629 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:08.255383 4629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:08.260847 4629 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:14:08.263669 4644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:08.265215 4629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:14:08.265396 4629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "9db9764d15a0471f9f2830e8c45237c3"
format_stamp: "Formatted at 2026-08-07 04:13:22 on dist-test-slave-8n49"
server_key: "a3569a85d330a6b99a7c98ad0b0b1545"
server_key_iv: "aac77176d9961b24eb7dd895845cbef4"
server_key_version: "encryptionkey@0"
I20260807 04:14:08.266422 4629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:08.281118 4629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:08.285400 4629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:08.285696 4629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:08.286621 4629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:08.287933 4629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:08.288054 4629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:08.288182 4629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:08.288288 4629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:08.322716 4629 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:32777
I20260807 04:14:08.322813 4762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:32777 every 8 connection(s)
I20260807 04:14:08.324220 4629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:14:08.328970 4765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.327166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:42631)
0807 04:14:08.327855 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:08.328077 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":273,"threads_started":1}
I20260807 04:14:08.329635 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 4629
W20260807 04:14:08.329677 4763 heartbeater.cc:644] Failed to heartbeat to 127.18.193.62:41531 (0 consecutive failures): Network error: Failed to ping master at 127.18.193.62:41531: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
I20260807 04:14:08.330538 4765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.330188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:51621)
0807 04:14:08.330361 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:08.330429 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 04:14:08.331444 4765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.331088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:59219)
0807 04:14:08.331235 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:08.331312 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 04:14:08.331701 4763 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 04:14:08.386396 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.385612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:51477)
0807 04:14:08.386087 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:08.386180 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":216,"threads_started":1}
I20260807 04:14:08.715330 4769 raft_consensus.cc:482] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:08.715662 4769 raft_consensus.cc:504] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:14:08.717386 4769 leader_election.cc:288] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411), 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777)
Aug 07 04:14:08 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076047, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
Aug 07 04:14:08 dist-test-slave-8n49 krb5kdc[3077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076047, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:14:08.740115 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.717781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:32777 (local address 127.18.193.1:41313)
0807 04:14:08.718202 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:08.718262 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:08.718968 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:08.723538 (+ 4570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:08.723552 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:08.724333 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:08.725122 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:08.725145 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.726922 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.726926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:08.728131 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:08.728143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.729288 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.729296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:08.729492 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:08.730343 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:08.730360 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:08.733107 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:08.736534 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:08.736545 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:08.736581 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:08.737004 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:08.737458 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:08.737465 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:08.737468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:08.737718 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:08.738536 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:08.738544 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:08.738892 (+ 348us) client_negotiation.cc:769] Sending connection context
0807 04:14:08.739165 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 04:14:08.739671 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":39}
I20260807 04:14:08.740181 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.718098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:41313 (local address 127.18.193.3:32777)
0807 04:14:08.720986 (+ 2888us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:08.721000 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:08.721048 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:14:08.722661 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:08.722670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:08.723064 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:08.723383 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:08.725333 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.726740 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.728278 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.729039 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.729274 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:08.733354 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:08.733362 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:08.733373 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:08.733410 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:08.736384 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:08.737167 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:08.737180 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:08.737192 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:08.737339 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:08.737849 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:08.737852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:08.737855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:08.738392 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:08.738594 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:08.739400 (+ 806us) server_negotiation.cc:299] Negotiation successful
0807 04:14:08.739780 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2675,"thread_start_us":2458,"threads_started":1}
I20260807 04:14:08.740432 4770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.717863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:45411 (local address 127.18.193.1:45133)
0807 04:14:08.721150 (+ 3287us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:08.721165 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:08.721315 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:08.723539 (+ 2224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:08.723551 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:08.724337 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:08.725150 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:08.725163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.726691 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.726699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:08.728142 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:08.728156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.729209 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.729217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:08.729499 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:08.730295 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:08.730320 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:08.732951 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:08.737030 (+ 4079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:08.737038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:08.737042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:08.737435 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:08.737972 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:08.737978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:08.737982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:08.738161 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:08.738933 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:08.738941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:08.739115 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 04:14:08.739297 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 04:14:08.739907 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3150,"mutex_wait_us":351,"spinlock_wait_cycles":31616,"thread_start_us":3003,"threads_started":1}
I20260807 04:14:08.741173 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.718220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:45133 (local address 127.18.193.2:45411)
0807 04:14:08.720303 (+ 2083us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:08.720325 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:08.721270 (+ 945us) server_negotiation.cc:407] Connection header received
0807 04:14:08.722789 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:08.722797 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:08.723115 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:08.723437 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:08.725354 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.726492 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.728318 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:08.728999 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:08.729180 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:08.733275 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:08.733284 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:08.733298 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:08.733340 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:08.736866 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:08.737641 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:08.737647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:08.737652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:08.737826 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:08.738295 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:08.738300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:08.738304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:08.738787 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:08.739031 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:08.739584 (+ 553us) server_negotiation.cc:299] Negotiation successful
0807 04:14:08.740035 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1825,"thread_start_us":1649,"threads_started":1}
W20260807 04:14:08.744231 4366 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
W20260807 04:14:08.744474 4364 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:14:08.744632 4364 leader_election.cc:302] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c63e16bfee648fe90400cdff5781fb1; no voters: 2412993129d64d75ad7bc8f852bbc0e5, 9db9764d15a0471f9f2830e8c45237c3
I20260807 04:14:08.745060 4769 raft_consensus.cc:2726] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:14:08.862767 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:08.861863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:49041)
0807 04:14:08.862372 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:08.862518 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":255,"threads_started":1}
I20260807 04:14:09.333124 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:09.332257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:57927)
0807 04:14:09.332803 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:09.332918 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":250,"threads_started":1}
I20260807 04:14:09.388192 4776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:09.387458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:36273)
0807 04:14:09.387902 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:09.388009 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":220,"threads_started":1}
I20260807 04:14:09.864485 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:09.863730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:57449)
0807 04:14:09.864228 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:09.864314 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":274,"threads_started":1}
I20260807 04:14:10.297511 4778 raft_consensus.cc:482] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:10.297703 4778 raft_consensus.cc:504] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:14:10.298324 4778 leader_election.cc:288] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411), 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777)
W20260807 04:14:10.299292 4364 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
W20260807 04:14:10.299484 4366 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:14:10.299651 4366 leader_election.cc:302] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c63e16bfee648fe90400cdff5781fb1; no voters: 2412993129d64d75ad7bc8f852bbc0e5, 9db9764d15a0471f9f2830e8c45237c3
I20260807 04:14:10.299921 4778 raft_consensus.cc:2726] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:14:10.334838 4779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:10.334123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:40903)
0807 04:14:10.334571 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:10.334655 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":233,"threads_started":1}
I20260807 04:14:10.389976 4780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:10.389129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:54897)
0807 04:14:10.389648 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:10.389760 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":259,"threads_started":1}
I20260807 04:14:10.866187 4781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:10.865465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:44945)
0807 04:14:10.865928 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:10.866015 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260807 04:14:11.336704 4782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:11.335836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:45871)
0807 04:14:11.336333 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:11.336436 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":259,"threads_started":1}
I20260807 04:14:11.391537 4783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:11.390867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:45407)
0807 04:14:11.391294 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:11.391375 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":231,"threads_started":1}
I20260807 04:14:11.867847 4784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:11.867118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:52813)
0807 04:14:11.867586 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:11.867668 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":234,"threads_started":1}
I20260807 04:14:12.338511 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:12.337697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:36579)
0807 04:14:12.338169 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:12.338258 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
I20260807 04:14:12.385506 4786 raft_consensus.cc:482] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:12.385689 4786 raft_consensus.cc:504] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:14:12.386425 4786 leader_election.cc:288] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411), 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777)
W20260807 04:14:12.387422 4366 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
W20260807 04:14:12.387598 4364 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:14:12.387735 4364 leader_election.cc:302] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c63e16bfee648fe90400cdff5781fb1; no voters: 2412993129d64d75ad7bc8f852bbc0e5, 9db9764d15a0471f9f2830e8c45237c3
I20260807 04:14:12.387972 4786 raft_consensus.cc:2726] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:14:12.393406 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:12.392568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:45023)
0807 04:14:12.393119 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:12.393210 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":242,"threads_started":1}
I20260807 04:14:12.870018 4788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:12.868950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:48395)
0807 04:14:12.869714 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:12.869811 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":408,"threads_started":1}
I20260807 04:14:13.340471 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:13.339685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:52275)
0807 04:14:13.340202 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:13.340303 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":285,"threads_started":1}
I20260807 04:14:13.395030 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:13.394302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:55689)
0807 04:14:13.394757 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:13.394842 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":244,"threads_started":1}
I20260807 04:14:13.872426 4792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:13.871181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:46839)
0807 04:14:13.872081 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:13.872177 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":620,"threads_started":1}
I20260807 04:14:14.342407 4794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:14.341592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:43697)
0807 04:14:14.342130 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:14.342223 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":286,"threads_started":1}
I20260807 04:14:14.396836 4795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:14.396017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:34349)
0807 04:14:14.396487 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:14.396633 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":245,"threads_started":1}
I20260807 04:14:14.874234 4796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:14.873390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:34391)
0807 04:14:14.873956 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:14.874041 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":261,"threads_started":1}
I20260807 04:14:15.344660 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:15.343576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:49283)
0807 04:14:15.344284 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:15.344367 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":366,"threads_started":1}
I20260807 04:14:15.398775 4798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:15.397912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:38363)
0807 04:14:15.398488 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:15.398583 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":316,"threads_started":1}
I20260807 04:14:15.876035 4799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:15.875299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:49385)
0807 04:14:15.875763 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:15.875847 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":254,"threads_started":1}
I20260807 04:14:16.346359 4800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:16.345596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:33487)
0807 04:14:16.346094 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:16.346179 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":280,"threads_started":1}
I20260807 04:14:16.400538 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:16.399750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:53931)
0807 04:14:16.400284 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:16.400369 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":319,"threads_started":1}
I20260807 04:14:16.877921 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:16.877081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:34881)
0807 04:14:16.877635 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:16.877740 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":262,"threads_started":1}
I20260807 04:14:16.903628 4803 raft_consensus.cc:482] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:16.903856 4803 raft_consensus.cc:504] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:14:16.904644 4803 leader_election.cc:288] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411), 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777)
W20260807 04:14:16.905648 4364 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
W20260807 04:14:16.905820 4366 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:14:16.905989 4366 leader_election.cc:302] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c63e16bfee648fe90400cdff5781fb1; no voters: 2412993129d64d75ad7bc8f852bbc0e5, 9db9764d15a0471f9f2830e8c45237c3
I20260807 04:14:16.906282 4803 raft_consensus.cc:2726] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:14:17.348232 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:17.347396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:57495)
0807 04:14:17.347957 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:17.348050 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":264,"threads_started":1}
I20260807 04:14:17.402522 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:17.401657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:35807)
0807 04:14:17.402186 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:17.402318 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":284,"threads_started":1}
I20260807 04:14:17.879849 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:17.879034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:50225)
0807 04:14:17.879529 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:17.879655 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":262,"threads_started":1}
I20260807 04:14:18.350307 4808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:18.349375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:46747)
0807 04:14:18.349930 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:18.350044 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":243,"threads_started":1}
I20260807 04:14:18.404167 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:18.403434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:36933)
0807 04:14:18.403882 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:18.403963 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
I20260807 04:14:18.881644 4810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:18.880892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:52485)
0807 04:14:18.881380 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:18.881477 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":270,"threads_started":1}
I20260807 04:14:19.352129 4811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:19.351398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:39257)
0807 04:14:19.351873 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:19.351957 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":231,"threads_started":1}
I20260807 04:14:19.405818 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:19.405061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:47291)
0807 04:14:19.405564 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:19.405648 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":250,"threads_started":1}
I20260807 04:14:19.883496 4813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:19.882708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:34873)
0807 04:14:19.883223 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:19.883314 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":266,"threads_started":1}
I20260807 04:14:20.353845 4814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:20.353120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:42725)
0807 04:14:20.353594 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:20.353674 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":249,"threads_started":1}
I20260807 04:14:20.407783 4815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:20.406858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:57253)
0807 04:14:20.407511 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:20.407604 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":372,"threads_started":1}
I20260807 04:14:20.885262 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:20.884450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:41195)
0807 04:14:20.884974 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:20.885084 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":307,"threads_started":1}
I20260807 04:14:20.971663 4817 raft_consensus.cc:482] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:20.971877 4817 raft_consensus.cc:504] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2412993129d64d75ad7bc8f852bbc0e5" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 45411 } } peers { permanent_uuid: "9c63e16bfee648fe90400cdff5781fb1" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 36177 } } peers { permanent_uuid: "9db9764d15a0471f9f2830e8c45237c3" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 32777 } }
I20260807 04:14:20.972527 4817 leader_election.cc:288] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411), 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777)
W20260807 04:14:20.973490 4366 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db9764d15a0471f9f2830e8c45237c3 (127.18.193.3:32777): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
W20260807 04:14:20.973690 4364 leader_election.cc:341] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2412993129d64d75ad7bc8f852bbc0e5 (127.18.193.2:45411): Not found: Tablet not found: b7488f18b72d447b8aeed6f8ec69f324
I20260807 04:14:20.973829 4364 leader_election.cc:302] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c63e16bfee648fe90400cdff5781fb1; no voters: 2412993129d64d75ad7bc8f852bbc0e5, 9db9764d15a0471f9f2830e8c45237c3
I20260807 04:14:20.974105 4817 raft_consensus.cc:2726] T b7488f18b72d447b8aeed6f8ec69f324 P 9c63e16bfee648fe90400cdff5781fb1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:14:21.355628 4818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:21.354863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:46499)
0807 04:14:21.355344 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:21.355445 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":244,"threads_started":1}
I20260807 04:14:21.409668 4820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:21.408828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:50179)
0807 04:14:21.409295 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:21.409404 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
I20260807 04:14:21.887012 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:21.886226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:58239)
0807 04:14:21.886700 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:21.886786 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":246,"threads_started":1}
I20260807 04:14:22.357609 4822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:22.356698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:51427)
0807 04:14:22.357264 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:22.357389 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":294,"threads_started":1}
I20260807 04:14:22.411600 4823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:22.410879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:60943)
0807 04:14:22.411317 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:22.411415 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":223,"threads_started":1}
I20260807 04:14:22.889084 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:22.888168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.2:39637)
0807 04:14:22.888759 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:22.888895 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":278,"threads_started":1}
I20260807 04:14:23.341601 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 4304
I20260807 04:14:23.359984 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:23.358849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.3:47727)
0807 04:14:23.359584 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:23.359757 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":346,"threads_started":1}
I20260807 04:14:23.364470 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:23.343148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.18.193.1:36177)
0807 04:14:23.343839 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:23.343851 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:23.343889 (+ 38us) server_negotiation.cc:407] Connection header received
0807 04:14:23.343981 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:23.343993 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:23.344281 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:23.344477 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:23.346033 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:23.347018 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:23.348307 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:23.348929 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:23.349072 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:23.357071 (+ 7999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:23.357086 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:23.357101 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:23.357145 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:23.361081 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:23.362011 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:23.362021 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:23.362024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:23.362230 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:23.362692 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:23.362700 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:23.362703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:23.363182 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:23.363481 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:23.363757 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 04:14:23.364058 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":292,"threads_started":1}
I20260807 04:14:23.365830 4478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:23.460117 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:23.458938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:41531 (local address 127.18.193.1:51643)
0807 04:14:23.459569 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:23.459723 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.62:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":287,"threads_started":1}
I20260807 04:14:23.460422 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 4304
I20260807 04:14:23.481945 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 4486
I20260807 04:14:23.496050 4829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:23.482731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.18.193.2:45411)
0807 04:14:23.483209 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:23.483214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:23.483232 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:14:23.483293 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:23.483297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:23.483358 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:23.483460 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:23.484873 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:23.485683 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:23.486948 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:23.487397 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:23.487525 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:23.490428 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:23.490433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:23.490437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:23.490475 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:23.493541 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:23.494278 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:23.494281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:23.494284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:23.494425 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:23.494767 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:23.494771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:23.494772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:23.495038 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:23.495254 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:23.495567 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 04:14:23.495747 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":271,"threads_started":1}
I20260807 04:14:23.496806 4620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:23.583101 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 4486
I20260807 04:14:23.603530 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 4629
I20260807 04:14:23.617637 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:23.604339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33956 (local address 127.18.193.3:32777)
0807 04:14:23.604972 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:23.604977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:23.604994 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:14:23.605067 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:23.605071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:23.605139 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:23.605232 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:23.606609 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:23.607423 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:23.608643 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:23.609058 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:23.609152 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:23.611709 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:23.611715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:23.611718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:23.611748 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:23.615139 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:23.615864 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:23.615868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:23.615870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:23.616005 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:23.616353 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:23.616356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:23.616358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:23.616667 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:23.616878 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:23.617178 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 04:14:23.617338 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":359,"threads_started":1}
I20260807 04:14:23.618350 4757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:23.703033 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 4629
2026-08-07T04:14:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64997 ms)
[----------] 30 tests from SecurityITest (314229 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:23 dist-test-slave-8n49 krb5kdc[4835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:23 dist-test-slave-8n49 krb5kdc[4835](info): set up 2 sockets
Aug 07 04:14:23 dist-test-slave-8n49 krb5kdc[4835](info): commencing operation
krb5kdc: starting...
W20260807 04:14:24.855681 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:25 dist-test-slave-8n49 krb5kdc[4835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:14:25.160375 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38775
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:38775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:25.508744 4849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:25.509122 4849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:25.509290 4849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:25.516609 4849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:25.516762 4849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:25.516871 4849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:25.516974 4849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:25.517066 4849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:25.526942 4849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:38775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:38775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:25.528359 4849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:25.529740 4849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:25.541925 4855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:25.542042 4858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:25.542069 4856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:25.542305 4849 server_base.cc:1034] running on GCE node
I20260807 04:14:25.543390 4849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:25.544870 4849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:25.546099 4849 hybrid_clock.cc:648] HybridClock initialized: now 1786076065546032 us; error 55 us; skew 500 ppm
I20260807 04:14:25.546439 4849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:25 dist-test-slave-8n49 krb5kdc[4835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076065, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:25.549557 4849 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:14:25.550994 4849 webserver.cc:460] Webserver started at http://127.18.193.62:33961/ using document root <none> and password file <none>
I20260807 04:14:25.551623 4849 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:25.551724 4849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:25.551981 4849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:25.555024 4849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "de28f212b8ce4f9a825fd03cb0847fe5"
format_stamp: "Formatted at 2026-08-07 04:14:25 on dist-test-slave-8n49"
server_key: "ac8cf88d4167077d4a898b3d8471689d"
server_key_iv: "a6e950d7e97edb8d74860ae8081ba05f"
server_key_version: "encryptionkey@0"
I20260807 04:14:25.555749 4849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "de28f212b8ce4f9a825fd03cb0847fe5"
format_stamp: "Formatted at 2026-08-07 04:14:25 on dist-test-slave-8n49"
server_key: "ac8cf88d4167077d4a898b3d8471689d"
server_key_iv: "a6e950d7e97edb8d74860ae8081ba05f"
server_key_version: "encryptionkey@0"
I20260807 04:14:25.560956 4849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:14:25.564283 4866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:25.565636 4849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:25.565814 4849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "de28f212b8ce4f9a825fd03cb0847fe5"
format_stamp: "Formatted at 2026-08-07 04:14:25 on dist-test-slave-8n49"
server_key: "ac8cf88d4167077d4a898b3d8471689d"
server_key_iv: "a6e950d7e97edb8d74860ae8081ba05f"
server_key_version: "encryptionkey@0"
I20260807 04:14:25.566004 4849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:25.578634 4849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:25.582744 4849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:25.583150 4849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:25.602063 4849 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:38775
I20260807 04:14:25.602097 4917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:38775 every 8 connection(s)
I20260807 04:14:25.603461 4849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:25.607272 4918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:25.609512 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 4849
I20260807 04:14:25.609745 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:25.610435 19204 external_mini_cluster.cc:1428] Setting key 86a6d2a76b4d2d5760a3a117ae5b42b7
I20260807 04:14:25.620609 4918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5: Bootstrap starting.
I20260807 04:14:25.624190 4918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:25.625269 4918 log.cc:824] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:14:25 dist-test-slave-8n49 krb5kdc[4835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:25.628468 4918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5: No bootstrap required, opened a new log
I20260807 04:14:25.635535 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:25.612517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.18.193.62:38775)
0807 04:14:25.613376 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:25.613392 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:25.613445 (+ 53us) server_negotiation.cc:407] Connection header received
0807 04:14:25.615844 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:25.615882 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:25.616366 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:25.616841 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:25.618176 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:25.620137 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:25.621762 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:25.622904 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:25.623081 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:25.627727 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:25.627736 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:25.627750 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:25.627792 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:25.631063 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:25.632036 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:25.632043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:25.632058 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:25.632259 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:25.632728 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:25.632732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:25.632734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:25.633306 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:25.633588 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:25.634036 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 04:14:25.634485 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":327,"threads_started":1}
I20260807 04:14:25.637507 4918 raft_consensus.cc:348] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38775 } }
I20260807 04:14:25.637897 4918 raft_consensus.cc:374] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:25.638015 4918 raft_consensus.cc:729] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de28f212b8ce4f9a825fd03cb0847fe5, State: Initialized, Role: FOLLOWER
I20260807 04:14:25.638605 4918 consensus_queue.cc:260] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38775 } }
I20260807 04:14:25.638847 4918 raft_consensus.cc:388] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:25.638991 4918 raft_consensus.cc:482] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:25.639173 4918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:25.641443 4918 raft_consensus.cc:504] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38775 } }
I20260807 04:14:25.641945 4918 leader_election.cc:302] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de28f212b8ce4f9a825fd03cb0847fe5; no voters:
I20260807 04:14:25.642560 4918 leader_election.cc:288] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:25.642711 4923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:25.644018 4923 raft_consensus.cc:686] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [term 1 LEADER]: Becoming Leader. State: Replica: de28f212b8ce4f9a825fd03cb0847fe5, State: Running, Role: LEADER
I20260807 04:14:25.644676 4923 consensus_queue.cc:237] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38775 } }
I20260807 04:14:25.644827 4918 sys_catalog.cc:564] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:25.647456 4925 sys_catalog.cc:455] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de28f212b8ce4f9a825fd03cb0847fe5. Latest consensus state: current_term: 1 leader_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38775 } } }
I20260807 04:14:25.647894 4925 sys_catalog.cc:458] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:25.648479 4924 sys_catalog.cc:455] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28f212b8ce4f9a825fd03cb0847fe5" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 38775 } } }
I20260807 04:14:25.648862 4924 sys_catalog.cc:458] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:25.657775 4932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:25.663661 4932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:25.672739 4932 catalog_manager.cc:1347] Generated new cluster ID: a77324a7a70b43bf99b641c0d3c12788
I20260807 04:14:25.672866 4932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:25.706944 4932 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:25.708859 4932 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:25.729676 4932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5: Generated new TSK 0
I20260807 04:14:25.730791 4932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:14:25.810300 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38775
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:26.159886 4949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:26.160166 4949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:26.160261 4949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:26.167673 4949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:26.167783 4949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:26.167896 4949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:14:26.178083 4949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:26.179374 4949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:26.180784 4949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:26.197114 4954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:26.197088 4957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:26.197162 4955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:26.197903 4949 server_base.cc:1034] running on GCE node
I20260807 04:14:26.198592 4949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:26.199626 4949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:26.200874 4949 hybrid_clock.cc:648] HybridClock initialized: now 1786076066200802 us; error 48 us; skew 500 ppm
I20260807 04:14:26.201216 4949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:26 dist-test-slave-8n49 krb5kdc[4835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076066, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:26.204469 4949 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:14:26.205951 4949 webserver.cc:460] Webserver started at http://127.18.193.1:37751/ using document root <none> and password file <none>
I20260807 04:14:26.206664 4949 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:26.206808 4949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:26.207157 4949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:26.210176 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "633ed9f4e7cc4e3197cb527a86970e7a"
format_stamp: "Formatted at 2026-08-07 04:14:26 on dist-test-slave-8n49"
server_key: "6f6c97c787601a182801c319e991fd3e"
server_key_iv: "ef8104b4ffbf4d8719b8bb25608a9778"
server_key_version: "encryptionkey@0"
I20260807 04:14:26.210981 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "633ed9f4e7cc4e3197cb527a86970e7a"
format_stamp: "Formatted at 2026-08-07 04:14:26 on dist-test-slave-8n49"
server_key: "6f6c97c787601a182801c319e991fd3e"
server_key_iv: "ef8104b4ffbf4d8719b8bb25608a9778"
server_key_version: "encryptionkey@0"
I20260807 04:14:26.216212 4949 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 04:14:26.219732 4964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:26.221117 4949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:26.221297 4949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "633ed9f4e7cc4e3197cb527a86970e7a"
format_stamp: "Formatted at 2026-08-07 04:14:26 on dist-test-slave-8n49"
server_key: "6f6c97c787601a182801c319e991fd3e"
server_key_iv: "ef8104b4ffbf4d8719b8bb25608a9778"
server_key_version: "encryptionkey@0"
I20260807 04:14:26.221500 4949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:26.243482 4949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:26.248034 4949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:26.248332 4949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:26.249428 4949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:26.250795 4949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:26.250903 4949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:26.251060 4949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:26.251129 4949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:26.284907 4949 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:34037
I20260807 04:14:26.284958 5079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:34037 every 8 connection(s)
I20260807 04:14:26.286250 4949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:14:26.289351 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 4949
I20260807 04:14:26.289568 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:14:26.290496 19204 external_mini_cluster.cc:1428] Setting key 4546bdedad4a3032022be933c3bbd714
Aug 07 04:14:26 dist-test-slave-8n49 krb5kdc[4835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076066, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:26.307649 5082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:26.288788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:34519 (local address 127.18.193.62:38775)
0807 04:14:26.289323 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:26.289330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:26.290489 (+ 1159us) server_negotiation.cc:407] Connection header received
0807 04:14:26.292533 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:26.292538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:26.292655 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:26.292773 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:26.294280 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.295025 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.296599 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.297015 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.297163 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:26.300904 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:26.300909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:26.300913 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:26.300945 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:26.303884 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:26.304793 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:26.304802 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:26.304809 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:26.304965 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:26.305398 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:26.305406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:26.305412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:26.305800 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:26.305960 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:26.306853 (+ 893us) server_negotiation.cc:299] Negotiation successful
0807 04:14:26.307164 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":267,"threads_started":1}
I20260807 04:14:26.308398 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:26.289345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38775 (local address 127.18.193.1:34519)
0807 04:14:26.290275 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:26.290338 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:26.292248 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:26.293026 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:26.293034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:26.293535 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:26.294101 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:26.294119 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.295196 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.295201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:26.296429 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:26.296442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.297261 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.297269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:26.297514 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:26.298230 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:26.298253 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:26.300681 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:26.304064 (+ 3383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:26.304072 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:26.304086 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:26.304647 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:26.305083 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:26.305087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:26.305089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:26.305280 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:26.305965 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:26.305973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:26.306357 (+ 384us) client_negotiation.cc:769] Sending connection context
0807 04:14:26.306672 (+ 315us) client_negotiation.cc:240] Negotiation successful
0807 04:14:26.307125 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":348,"threads_started":1}
I20260807 04:14:26.310254 5080 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38775
I20260807 04:14:26.310592 5080 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:26.311652 5080 heartbeater.cc:511] Master 127.18.193.62:38775 requested a full tablet report, sending...
I20260807 04:14:26.314100 4883 ts_manager.cc:194] Registered new tserver with Master: 633ed9f4e7cc4e3197cb527a86970e7a (127.18.193.1:34037)
I20260807 04:14:26.315622 4883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:34519
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:14:26.357808 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38775
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:26.696477 5086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:26.696924 5086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:26.697103 5086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:26.704734 5086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:26.704890 5086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:26.705080 5086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:14:26.715051 5086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:26.716408 5086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:26.717760 5086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:26.731549 5093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:26.731632 5092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:26.731606 5095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:26.732476 5086 server_base.cc:1034] running on GCE node
I20260807 04:14:26.733130 5086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:26.734141 5086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:26.735375 5086 hybrid_clock.cc:648] HybridClock initialized: now 1786076066735324 us; error 56 us; skew 500 ppm
I20260807 04:14:26.735736 5086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:26 dist-test-slave-8n49 krb5kdc[4835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076066, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:26.739369 5086 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:14:26.741008 5086 webserver.cc:460] Webserver started at http://127.18.193.2:41879/ using document root <none> and password file <none>
I20260807 04:14:26.741758 5086 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:26.741917 5086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:26.742260 5086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:26.745666 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "446e7dfd1aa041b8ac33a15ecd82bdef"
format_stamp: "Formatted at 2026-08-07 04:14:26 on dist-test-slave-8n49"
server_key: "65da315df476803e82264abe93205604"
server_key_iv: "d1b9f83c6794cfb30e248d3adda972e5"
server_key_version: "encryptionkey@0"
I20260807 04:14:26.746546 5086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "446e7dfd1aa041b8ac33a15ecd82bdef"
format_stamp: "Formatted at 2026-08-07 04:14:26 on dist-test-slave-8n49"
server_key: "65da315df476803e82264abe93205604"
server_key_iv: "d1b9f83c6794cfb30e248d3adda972e5"
server_key_version: "encryptionkey@0"
I20260807 04:14:26.752178 5086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:14:26.756270 5102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:26.757740 5086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:26.757911 5086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "446e7dfd1aa041b8ac33a15ecd82bdef"
format_stamp: "Formatted at 2026-08-07 04:14:26 on dist-test-slave-8n49"
server_key: "65da315df476803e82264abe93205604"
server_key_iv: "d1b9f83c6794cfb30e248d3adda972e5"
server_key_version: "encryptionkey@0"
I20260807 04:14:26.758158 5086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:26.776942 5086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:26.781549 5086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:26.781853 5086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:26.782860 5086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:26.784273 5086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:26.784373 5086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:26.784519 5086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:26.784619 5086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 04:14:26.819303 5086 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:44857
I20260807 04:14:26.819350 5226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:44857 every 8 connection(s)
I20260807 04:14:26.820639 5086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:14:26.826673 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 5086
I20260807 04:14:26.826875 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:14:26.827332 19204 external_mini_cluster.cc:1428] Setting key 4ff01b77de5caa14a80c6094b90a7c2e
Aug 07 04:14:26 dist-test-slave-8n49 krb5kdc[4835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076066, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:26.841334 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:26.823131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:40729 (local address 127.18.193.62:38775)
0807 04:14:26.823607 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:26.823611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:26.824361 (+ 750us) server_negotiation.cc:407] Connection header received
0807 04:14:26.826386 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:26.826391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:26.826482 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:26.826612 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:26.828194 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.829254 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.830781 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.831212 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.831335 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:26.835008 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:26.835013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:26.835016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:26.835048 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:26.837703 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:26.838661 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:26.838665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:26.838668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:26.838799 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:26.839328 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:26.839332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:26.839334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:26.839608 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:26.839734 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:26.840605 (+ 871us) server_negotiation.cc:299] Negotiation successful
0807 04:14:26.840874 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":256,"threads_started":1}
I20260807 04:14:26.841996 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:26.823497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38775 (local address 127.18.193.2:40729)
0807 04:14:26.824198 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:26.824241 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:26.826117 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:26.826883 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:26.826892 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:26.827440 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:26.828014 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:26.828032 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.829417 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.829421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:26.830639 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:26.830651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:26.831444 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:26.831452 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:26.831672 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:26.832353 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:26.832392 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:26.834763 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:26.837864 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:26.837876 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:26.837895 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:26.838537 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:26.838911 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:26.838918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:26.838924 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:26.839185 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:26.839746 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:26.839756 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:26.840171 (+ 415us) client_negotiation.cc:769] Sending connection context
0807 04:14:26.840404 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 04:14:26.840855 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":272,"threads_started":1}
I20260807 04:14:26.843526 5227 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38775
I20260807 04:14:26.843848 5227 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:26.844645 5227 heartbeater.cc:511] Master 127.18.193.62:38775 requested a full tablet report, sending...
I20260807 04:14:26.846328 4883 ts_manager.cc:194] Registered new tserver with Master: 446e7dfd1aa041b8ac33a15ecd82bdef (127.18.193.2:44857)
I20260807 04:14:26.847229 4883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:40729
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:14:26.897229 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38775
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:27.235886 5233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:27.236181 5233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:27.236347 5233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:27.243629 5233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:27.243752 5233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:27.243949 5233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:14:27.254233 5233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:38775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:27.255764 5233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:27.257246 5233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:27.270889 5240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:27.271102 5241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:27.271206 5243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:27.272197 5233 server_base.cc:1034] running on GCE node
I20260807 04:14:27.272923 5233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:27.273917 5233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:27.275162 5233 hybrid_clock.cc:648] HybridClock initialized: now 1786076067275113 us; error 58 us; skew 500 ppm
I20260807 04:14:27.275494 5233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:27 dist-test-slave-8n49 krb5kdc[4835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076067, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:27.278712 5233 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:14:27.280213 5233 webserver.cc:460] Webserver started at http://127.18.193.3:40427/ using document root <none> and password file <none>
I20260807 04:14:27.280980 5233 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:27.281127 5233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:27.281458 5233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:27.284641 5233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "b477b9026e8a4e1fa29e63f6c6710f05"
format_stamp: "Formatted at 2026-08-07 04:14:27 on dist-test-slave-8n49"
server_key: "8a3c24c73a53026b2a158897d7aeb548"
server_key_iv: "5893b525e1adb6996688247707b99997"
server_key_version: "encryptionkey@0"
I20260807 04:14:27.285483 5233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "b477b9026e8a4e1fa29e63f6c6710f05"
format_stamp: "Formatted at 2026-08-07 04:14:27 on dist-test-slave-8n49"
server_key: "8a3c24c73a53026b2a158897d7aeb548"
server_key_iv: "5893b525e1adb6996688247707b99997"
server_key_version: "encryptionkey@0"
I20260807 04:14:27.290725 5233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 04:14:27.294168 5250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:27.295490 5233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:14:27.295661 5233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "b477b9026e8a4e1fa29e63f6c6710f05"
format_stamp: "Formatted at 2026-08-07 04:14:27 on dist-test-slave-8n49"
server_key: "8a3c24c73a53026b2a158897d7aeb548"
server_key_iv: "5893b525e1adb6996688247707b99997"
server_key_version: "encryptionkey@0"
I20260807 04:14:27.295864 5233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:27.319551 5080 heartbeater.cc:503] Master 127.18.193.62:38775 was elected leader, sending a full tablet report...
I20260807 04:14:27.319784 5233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:27.324316 5233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:27.324647 5233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:27.325747 5233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:27.327171 5233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:27.327273 5233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:27.327423 5233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:27.327492 5233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:27.361552 5233 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:42199
I20260807 04:14:27.361618 5374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:42199 every 8 connection(s)
I20260807 04:14:27.363037 5233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:14:27.366896 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 5233
I20260807 04:14:27.367084 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:14:27.367596 19204 external_mini_cluster.cc:1428] Setting key a0160eed10792841003fa2bdfd849f62
Aug 07 04:14:27 dist-test-slave-8n49 krb5kdc[4835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076067, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:27.385717 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.365575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:40205 (local address 127.18.193.62:38775)
0807 04:14:27.366006 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.366011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.367029 (+ 1018us) server_negotiation.cc:407] Connection header received
0807 04:14:27.369160 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.369165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.369252 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.369375 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:27.371447 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.372223 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.373842 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.374487 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.374627 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.379122 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:27.379129 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:27.379134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:27.379175 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:27.382526 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.383414 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.383419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.383422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.383572 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.384079 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.384084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.384088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.384377 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:27.384501 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.385258 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.385427 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":196,"threads_started":1}
I20260807 04:14:27.386713 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.366067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:38775 (local address 127.18.193.3:40205)
0807 04:14:27.366774 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:27.366811 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:27.368826 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:27.369760 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:27.369774 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:27.370467 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:27.371277 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.371299 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.372375 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.372385 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.373658 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.373670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.374693 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.374699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.374975 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.375787 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:27.375838 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:27.378867 (+ 3029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:27.382701 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:27.382711 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:27.382729 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:27.383291 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:27.383687 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:27.383695 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:27.383699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:27.383950 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:27.384507 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:27.384514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:27.384878 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 04:14:27.385136 (+ 258us) client_negotiation.cc:240] Negotiation successful
0807 04:14:27.385542 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":261,"threads_started":1}
I20260807 04:14:27.388367 5375 heartbeater.cc:347] Connected to a master server at 127.18.193.62:38775
I20260807 04:14:27.388819 5375 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:27.389582 5375 heartbeater.cc:511] Master 127.18.193.62:38775 requested a full tablet report, sending...
I20260807 04:14:27.391129 4883 ts_manager.cc:194] Registered new tserver with Master: b477b9026e8a4e1fa29e63f6c6710f05 (127.18.193.3:42199)
I20260807 04:14:27.391979 4883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:40205
I20260807 04:14:27.396289 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:14:27.414672 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.401635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50052 (local address 127.18.193.62:38775)
0807 04:14:27.401825 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.401830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.402263 (+ 433us) server_negotiation.cc:407] Connection header received
0807 04:14:27.402435 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.402439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.402524 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.402627 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:27.404083 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.404962 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.406291 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.406709 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.406809 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.409441 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:27.409446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:27.409449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:27.409484 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:27.412160 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.412960 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.412965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.412967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.413100 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.413432 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.413435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.413437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.413761 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:27.413945 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.414249 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.414422 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:14:27.418777 4883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:14:27.425834 4883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:14:27.463987 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.438198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44857 (local address 127.0.0.1:54580)
0807 04:14:27.454258 (+ 16060us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:27.454282 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:27.454458 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:27.455560 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:27.455569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:27.455606 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:27.456021 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.456039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.458205 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.458210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.460720 (+ 2510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.460771 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.462970 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.462977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.463170 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.463223 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 04:14:27.463352 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 04:14:27.463542 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15883,"mutex_wait_us":116,"thread_start_us":16076,"threads_started":1}
I20260807 04:14:27.464054 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.438800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.18.193.2:44857)
0807 04:14:27.453690 (+ 14890us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.453701 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.454608 (+ 907us) server_negotiation.cc:407] Connection header received
0807 04:14:27.454730 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.454738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.455009 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.455232 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:27.456307 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.458032 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.461021 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.462473 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.462611 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.462820 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.463453 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.463642 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14712,"thread_start_us":14556,"threads_started":1}
I20260807 04:14:27.465422 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.438068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:42199 (local address 127.0.0.1:33064)
0807 04:14:27.450571 (+ 12503us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:27.450630 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:27.450889 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:27.456509 (+ 5620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:27.456513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:27.456538 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:27.457026 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.457044 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.459060 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.459067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.462218 (+ 3151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.462233 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.464423 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.464432 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.464691 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.464743 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 04:14:27.464842 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 04:14:27.464983 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12139,"mutex_wait_us":117,"thread_start_us":13066,"threads_started":1}
I20260807 04:14:27.466342 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.438354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.18.193.3:42199)
0807 04:14:27.456001 (+ 17647us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.456009 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.456041 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:14:27.456137 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.456147 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.456322 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.456507 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:27.457197 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.458905 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.462417 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.463861 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.463997 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.464268 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.465753 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.465973 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17452,"thread_start_us":17269,"threads_started":1}
I20260807 04:14:27.466538 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.439176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.18.193.1:34037)
0807 04:14:27.453210 (+ 14034us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.453224 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.455017 (+ 1793us) server_negotiation.cc:407] Connection header received
0807 04:14:27.455152 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.455162 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.455453 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.455660 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:27.456967 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.458757 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.462678 (+ 3921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.464155 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.464289 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.464511 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.466043 (+ 1532us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.466214 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13810,"thread_start_us":13617,"threads_started":1}
I20260807 04:14:27.465567 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.438603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:34037 (local address 127.0.0.1:50316)
0807 04:14:27.454673 (+ 16070us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:27.454699 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:27.454864 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:27.455930 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:27.455935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:27.455968 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:27.456701 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.456719 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.458932 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.458937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.462521 (+ 3584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.462534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.464691 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.464699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.464899 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.464932 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:14:27.465013 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 04:14:27.465139 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15898,"mutex_wait_us":324,"thread_start_us":15286,"threads_started":1}
I20260807 04:14:27.468323 5153 tablet_service.cc:1467] Processing CreateTablet for tablet 0683c75bac3c43229e599ac6e67171bb (DEFAULT_TABLE table=test-table [id=56a7e917300f4998b97091070115e56d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:27.469825 5153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0683c75bac3c43229e599ac6e67171bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:27.470345 5303 tablet_service.cc:1467] Processing CreateTablet for tablet 0683c75bac3c43229e599ac6e67171bb (DEFAULT_TABLE table=test-table [id=56a7e917300f4998b97091070115e56d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:27.472019 5303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0683c75bac3c43229e599ac6e67171bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:27.472231 5014 tablet_service.cc:1467] Processing CreateTablet for tablet 0683c75bac3c43229e599ac6e67171bb (DEFAULT_TABLE table=test-table [id=56a7e917300f4998b97091070115e56d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:27.473740 5014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0683c75bac3c43229e599ac6e67171bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:27.481220 5395 tablet_bootstrap.cc:492] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Bootstrap starting.
I20260807 04:14:27.484936 5395 tablet_bootstrap.cc:654] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:27.485956 5395 log.cc:824] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:27.488528 5395 tablet_bootstrap.cc:492] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: No bootstrap required, opened a new log
I20260807 04:14:27.488847 5395 ts_tablet_manager.cc:1397] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260807 04:14:27.489675 5398 tablet_bootstrap.cc:492] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Bootstrap starting.
I20260807 04:14:27.491493 5397 tablet_bootstrap.cc:492] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Bootstrap starting.
I20260807 04:14:27.495141 5397 tablet_bootstrap.cc:654] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:27.495173 5398 tablet_bootstrap.cc:654] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:27.496174 5398 log.cc:824] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:27.496325 5397 log.cc:824] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:27.499486 5398 tablet_bootstrap.cc:492] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: No bootstrap required, opened a new log
I20260807 04:14:27.499812 5398 ts_tablet_manager.cc:1397] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 04:14:27.499775 5395 raft_consensus.cc:348] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.500367 5395 raft_consensus.cc:374] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:27.500499 5395 raft_consensus.cc:729] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 446e7dfd1aa041b8ac33a15ecd82bdef, State: Initialized, Role: FOLLOWER
I20260807 04:14:27.500597 5397 tablet_bootstrap.cc:492] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: No bootstrap required, opened a new log
I20260807 04:14:27.500844 5397 ts_tablet_manager.cc:1397] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 04:14:27.501180 5395 consensus_queue.cc:260] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.502342 5227 heartbeater.cc:503] Master 127.18.193.62:38775 was elected leader, sending a full tablet report...
I20260807 04:14:27.502959 5395 ts_tablet_manager.cc:1428] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260807 04:14:27.509260 5397 raft_consensus.cc:348] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.509636 5397 raft_consensus.cc:374] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:27.509749 5397 raft_consensus.cc:729] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b477b9026e8a4e1fa29e63f6c6710f05, State: Initialized, Role: FOLLOWER
I20260807 04:14:27.510378 5397 consensus_queue.cc:260] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.510632 5398 raft_consensus.cc:348] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.511160 5398 raft_consensus.cc:374] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:27.511271 5398 raft_consensus.cc:729] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 633ed9f4e7cc4e3197cb527a86970e7a, State: Initialized, Role: FOLLOWER
I20260807 04:14:27.511688 5375 heartbeater.cc:503] Master 127.18.193.62:38775 was elected leader, sending a full tablet report...
I20260807 04:14:27.511901 5398 consensus_queue.cc:260] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.512221 5397 ts_tablet_manager.cc:1428] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260807 04:14:27.513650 5398 ts_tablet_manager.cc:1428] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 04:14:27.524283 5404 raft_consensus.cc:482] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:27.524627 5404 raft_consensus.cc:504] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.526387 5404 leader_election.cc:288] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b477b9026e8a4e1fa29e63f6c6710f05 (127.18.193.3:42199), 446e7dfd1aa041b8ac33a15ecd82bdef (127.18.193.2:44857)
I20260807 04:14:27.536959 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.526866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:54929 (local address 127.18.193.3:42199)
0807 04:14:27.527296 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.527303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.528083 (+ 780us) server_negotiation.cc:407] Connection header received
0807 04:14:27.528347 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.528353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.528459 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.528614 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:27.529400 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.530740 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.534190 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.535578 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.535763 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.535841 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.536491 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.536665 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
I20260807 04:14:27.536959 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.527136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:54659 (local address 127.18.193.2:44857)
0807 04:14:27.527378 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.527387 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.528306 (+ 919us) server_negotiation.cc:407] Connection header received
0807 04:14:27.528507 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.528516 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.528644 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.528742 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:27.529534 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.530740 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.534187 (+ 3447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.535589 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.535759 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.535841 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.536516 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.536681 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 04:14:27.537298 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.527313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44857 (local address 127.18.193.1:54659)
0807 04:14:27.528193 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:27.528213 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:27.528335 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:27.528771 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:27.528777 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:27.528823 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:27.529312 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.529326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.530907 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.530912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.533986 (+ 3074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.534004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.535879 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.535884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.536199 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.536340 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 04:14:27.536444 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 04:14:27.536682 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":385,"thread_start_us":515,"threads_started":1}
I20260807 04:14:27.537511 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.526853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:42199 (local address 127.18.193.1:54929)
0807 04:14:27.527938 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:27.527963 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:27.528151 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:27.528652 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:27.528656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:27.528679 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:27.529206 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.529220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.530906 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.530912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.533997 (+ 3085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:27.534006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.535862 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.535870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:27.536198 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.536336 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 04:14:27.536453 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 04:14:27.536682 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":266,"thread_start_us":630,"threads_started":1}
W20260807 04:14:27.538693 5081 tablet.cc:2369] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:27.538738 5177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0683c75bac3c43229e599ac6e67171bb" candidate_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" is_pre_election: true
I20260807 04:14:27.538738 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0683c75bac3c43229e599ac6e67171bb" candidate_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" is_pre_election: true
I20260807 04:14:27.539373 5324 raft_consensus.cc:2454] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 633ed9f4e7cc4e3197cb527a86970e7a in term 0.
I20260807 04:14:27.539373 5177 raft_consensus.cc:2454] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 633ed9f4e7cc4e3197cb527a86970e7a in term 0.
I20260807 04:14:27.540169 4968 leader_election.cc:302] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 633ed9f4e7cc4e3197cb527a86970e7a, b477b9026e8a4e1fa29e63f6c6710f05; no voters:
I20260807 04:14:27.540644 5404 raft_consensus.cc:2781] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:14:27.540812 5404 raft_consensus.cc:482] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:14:27.540932 5404 raft_consensus.cc:3037] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:27.543260 5404 raft_consensus.cc:504] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.543906 5404 leader_election.cc:288] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [CANDIDATE]: Term 1 election: Requested vote from peers b477b9026e8a4e1fa29e63f6c6710f05 (127.18.193.3:42199), 446e7dfd1aa041b8ac33a15ecd82bdef (127.18.193.2:44857)
I20260807 04:14:27.544354 5324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0683c75bac3c43229e599ac6e67171bb" candidate_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b477b9026e8a4e1fa29e63f6c6710f05"
I20260807 04:14:27.544500 5177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0683c75bac3c43229e599ac6e67171bb" candidate_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef"
I20260807 04:14:27.544623 5324 raft_consensus.cc:3037] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:27.544763 5177 raft_consensus.cc:3037] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:27.548292 5324 raft_consensus.cc:2454] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633ed9f4e7cc4e3197cb527a86970e7a in term 1.
I20260807 04:14:27.548285 5177 raft_consensus.cc:2454] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633ed9f4e7cc4e3197cb527a86970e7a in term 1.
I20260807 04:14:27.548789 4966 leader_election.cc:302] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 446e7dfd1aa041b8ac33a15ecd82bdef, 633ed9f4e7cc4e3197cb527a86970e7a, b477b9026e8a4e1fa29e63f6c6710f05; no voters:
I20260807 04:14:27.549145 5404 raft_consensus.cc:2781] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:27.549690 5404 raft_consensus.cc:686] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 1 LEADER]: Becoming Leader. State: Replica: 633ed9f4e7cc4e3197cb527a86970e7a, State: Running, Role: LEADER
I20260807 04:14:27.550174 5404 consensus_queue.cc:237] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } }
I20260807 04:14:27.554823 4883 catalog_manager.cc:5495] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 633ed9f4e7cc4e3197cb527a86970e7a (127.18.193.1). New cstate: current_term: 1 leader_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "633ed9f4e7cc4e3197cb527a86970e7a" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 34037 } health_report { overall_health: HEALTHY } } }
W20260807 04:14:27.572676 5228 tablet.cc:2369] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:27.573623 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.567695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.18.193.1:34037)
0807 04:14:27.567887 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.567892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.568017 (+ 125us) server_negotiation.cc:407] Connection header received
0807 04:14:27.568207 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.568211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.568315 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.568454 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:14:27.568916 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.569698 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.571275 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.571722 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.571826 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.572275 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:14:27.572951 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:14:27.573121 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.573235 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.573325 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:14:27.582103 5177 raft_consensus.cc:1261] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 1 FOLLOWER]: Refusing update from remote peer 633ed9f4e7cc4e3197cb527a86970e7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:27.582108 5324 raft_consensus.cc:1261] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 1 FOLLOWER]: Refusing update from remote peer 633ed9f4e7cc4e3197cb527a86970e7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:27.583196 5407 consensus_queue.cc:1035] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "446e7dfd1aa041b8ac33a15ecd82bdef" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:27.583592 5404 consensus_queue.cc:1035] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b477b9026e8a4e1fa29e63f6c6710f05" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 42199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:27.606007 5411 mvcc.cc:205] Tried to move back new op lower bound from 7315767572805713920 to 7315767572689309696. Current Snapshot: MvccSnapshot[applied={T|T < 7315767572805713920}]
I20260807 04:14:27.616606 4882 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:27.617014 4882 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:27.621367 4882 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de28f212b8ce4f9a825fd03cb0847fe5: Sending DeleteTablet for 3 replicas of tablet 0683c75bac3c43229e599ac6e67171bb
I20260807 04:14:27.622583 5303 tablet_service.cc:1514] Processing DeleteTablet for tablet 0683c75bac3c43229e599ac6e67171bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:27 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:33064
I20260807 04:14:27.622593 5153 tablet_service.cc:1514] Processing DeleteTablet for tablet 0683c75bac3c43229e599ac6e67171bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:27 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:54580
I20260807 04:14:27.623374 5014 tablet_service.cc:1514] Processing DeleteTablet for tablet 0683c75bac3c43229e599ac6e67171bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:27 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:50316
I20260807 04:14:27.624758 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 4949
I20260807 04:14:27.624796 5421 tablet_replica.cc:330] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: stopping tablet replica
I20260807 04:14:27.625430 5421 raft_consensus.cc:2229] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:27.626221 5421 raft_consensus.cc:2258] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:27.627168 5420 tablet_replica.cc:330] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: stopping tablet replica
I20260807 04:14:27.627622 5420 raft_consensus.cc:2229] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:27.628208 5420 raft_consensus.cc:2258] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:27.628368 5422 tablet_replica.cc:330] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: stopping tablet replica
I20260807 04:14:27.628783 5422 raft_consensus.cc:2229] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:14:27.629463 5422 raft_consensus.cc:2258] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:27.629802 5421 ts_tablet_manager.cc:1905] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:27.630687 5420 ts_tablet_manager.cc:1905] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:27.637260 5422 ts_tablet_manager.cc:1905] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:27.637802 5421 ts_tablet_manager.cc:1918] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:27.638019 5421 log.cc:1196] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/0683c75bac3c43229e599ac6e67171bb
I20260807 04:14:27.638067 5420 ts_tablet_manager.cc:1918] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:27.638382 5420 log.cc:1196] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/0683c75bac3c43229e599ac6e67171bb
I20260807 04:14:27.638629 5421 ts_tablet_manager.cc:1939] T 0683c75bac3c43229e599ac6e67171bb P b477b9026e8a4e1fa29e63f6c6710f05: Deleting consensus metadata
I20260807 04:14:27.639047 5420 ts_tablet_manager.cc:1939] T 0683c75bac3c43229e599ac6e67171bb P 446e7dfd1aa041b8ac33a15ecd82bdef: Deleting consensus metadata
I20260807 04:14:27.640910 4868 catalog_manager.cc:4841] TS 446e7dfd1aa041b8ac33a15ecd82bdef (127.18.193.2:44857): tablet 0683c75bac3c43229e599ac6e67171bb (table test-table [id=56a7e917300f4998b97091070115e56d]) successfully deleted
I20260807 04:14:27.640856 4870 catalog_manager.cc:4841] TS b477b9026e8a4e1fa29e63f6c6710f05 (127.18.193.3:42199): tablet 0683c75bac3c43229e599ac6e67171bb (table test-table [id=56a7e917300f4998b97091070115e56d]) successfully deleted
I20260807 04:14:27.644969 5422 ts_tablet_manager.cc:1918] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:27.645181 5422 log.cc:1196] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/0683c75bac3c43229e599ac6e67171bb
I20260807 04:14:27.645720 5422 ts_tablet_manager.cc:1939] T 0683c75bac3c43229e599ac6e67171bb P 633ed9f4e7cc4e3197cb527a86970e7a: Deleting consensus metadata
Aug 07 04:14:27 dist-test-slave-8n49 krb5kdc[4835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:14:27.647619 4869 catalog_manager.cc:4841] TS 633ed9f4e7cc4e3197cb527a86970e7a (127.18.193.1:34037): tablet 0683c75bac3c43229e599ac6e67171bb (table test-table [id=56a7e917300f4998b97091070115e56d]) successfully deleted
I20260807 04:14:27.655332 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.627167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50328 (local address 127.18.193.1:34037)
0807 04:14:27.627351 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.627358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.628710 (+ 1352us) server_negotiation.cc:407] Connection header received
0807 04:14:27.629027 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.629035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.629131 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.629221 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:27.631572 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.633015 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.640814 (+ 7799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.641598 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.641714 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.647866 (+ 6152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:27.647880 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:27.647899 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:27.647948 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:27.652076 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.652973 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.652983 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.652987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.653183 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.653610 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.653616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.653619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.654086 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:27.654393 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.654697 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.654937 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 04:14:27.656111 5074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:27.756712 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 4949
I20260807 04:14:27.779165 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 5086
Aug 07 04:14:27 dist-test-slave-8n49 krb5kdc[4835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:14:27.795313 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.779898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54590 (local address 127.18.193.2:44857)
0807 04:14:27.780075 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.780080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.780166 (+ 86us) server_negotiation.cc:407] Connection header received
0807 04:14:27.780328 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.780332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.780405 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.780495 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:27.782034 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.782953 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.784518 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.785009 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.785112 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.789259 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:27.789274 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:27.789289 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:27.789328 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:27.792237 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.792992 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.792999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.793002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.793131 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.793676 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.793682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.793686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.794301 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:27.794587 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.794864 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.795052 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:14:27.796067 5222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:27.886315 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 5086
I20260807 04:14:27.905674 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 5233
Aug 07 04:14:27 dist-test-slave-8n49 krb5kdc[4835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:14:27.921164 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:27.906552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33078 (local address 127.18.193.3:42199)
0807 04:14:27.906842 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:27.906850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:27.906871 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:14:27.907054 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:27.907059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:27.907124 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:27.907230 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:27.908601 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.909324 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.910620 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:27.911031 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:27.911163 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:27.915534 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:27.915550 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:27.915567 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:27.915606 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:27.918214 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.918879 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.918887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.918889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.919042 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:27.919421 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:27.919427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:27.919430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:27.919774 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:27.920082 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:27.920413 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 04:14:27.920838 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 04:14:27.921921 5370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:28.012926 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 5233
I20260807 04:14:28.031924 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 4849
I20260807 04:14:28.033193 4913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:28.161751 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 4849
2026-08-07T04:14:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4466 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T04:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:28Z Disabled control of system clock
I20260807 04:14:28.254694 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34961
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:34961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:14:28.610299 5434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:28.610630 5434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:28.610715 5434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:28.618603 5434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:28.618741 5434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:28.618782 5434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:28.618834 5434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:28.618880 5434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:28.628583 5434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:34961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:28.629880 5434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:28.631637 5434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:28.644358 5443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:28.644454 5441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:28.644923 5440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:28.645313 5434 server_base.cc:1034] running on GCE node
I20260807 04:14:28.646078 5434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:28.647451 5434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:28.648658 5434 hybrid_clock.cc:648] HybridClock initialized: now 1786076068648605 us; error 52 us; skew 500 ppm
I20260807 04:14:28.649020 5434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:28.651242 5434 webserver.cc:460] Webserver started at http://127.18.193.62:39245/ using document root <none> and password file <none>
I20260807 04:14:28.651926 5434 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:28.652065 5434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:28.652417 5434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:28.655377 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "1b212360f88249d497fa7877bd01fe0d"
format_stamp: "Formatted at 2026-08-07 04:14:28 on dist-test-slave-8n49"
server_key: "a9caada5b68ae981e4c6bc26599ad0bf"
server_key_iv: "0d314ce651381afcf5598284bc52ecf6"
server_key_version: "encryptionkey@0"
I20260807 04:14:28.656229 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "1b212360f88249d497fa7877bd01fe0d"
format_stamp: "Formatted at 2026-08-07 04:14:28 on dist-test-slave-8n49"
server_key: "a9caada5b68ae981e4c6bc26599ad0bf"
server_key_iv: "0d314ce651381afcf5598284bc52ecf6"
server_key_version: "encryptionkey@0"
I20260807 04:14:28.661640 5434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:14:28.665305 5449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:28.666761 5434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:14:28.666962 5434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "1b212360f88249d497fa7877bd01fe0d"
format_stamp: "Formatted at 2026-08-07 04:14:28 on dist-test-slave-8n49"
server_key: "a9caada5b68ae981e4c6bc26599ad0bf"
server_key_iv: "0d314ce651381afcf5598284bc52ecf6"
server_key_version: "encryptionkey@0"
I20260807 04:14:28.667207 5434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:28.688165 5434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:28.689064 5434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:28.689359 5434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:28.709234 5434 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:34961
I20260807 04:14:28.709194 5504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:34961 every 8 connection(s)
I20260807 04:14:28.710584 5434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:28.713558 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 5434
I20260807 04:14:28.713763 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:28.714385 19204 external_mini_cluster.cc:1428] Setting key 83e0878f9ca0c3abceec960c73b0fa95
I20260807 04:14:28.714717 5505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:28.729446 5505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d: Bootstrap starting.
I20260807 04:14:28.729987 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:28.716461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.18.193.62:34961)
0807 04:14:28.717684 (+ 1223us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:28.717695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:28.717738 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:14:28.719787 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:28.719808 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:28.720241 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:28.720776 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:28.721346 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:28.722737 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:28.724954 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:28.725857 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:28.725993 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:28.727503 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:28.727511 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:28.727524 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:28.728084 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:28.728174 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:28.728201 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:28.728309 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:28.728545 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 04:14:28.728959 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":377,"threads_started":1}
I20260807 04:14:28.732704 5505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:28.733686 5505 log.cc:824] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:28.736653 5505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d: No bootstrap required, opened a new log
I20260807 04:14:28.745563 5505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b212360f88249d497fa7877bd01fe0d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34961 } }
I20260807 04:14:28.746068 5505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:28.746184 5505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b212360f88249d497fa7877bd01fe0d, State: Initialized, Role: FOLLOWER
I20260807 04:14:28.747009 5505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b212360f88249d497fa7877bd01fe0d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34961 } }
I20260807 04:14:28.747318 5505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:28.747491 5505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:28.747687 5505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:28.750329 5505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b212360f88249d497fa7877bd01fe0d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34961 } }
I20260807 04:14:28.750909 5505 leader_election.cc:302] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b212360f88249d497fa7877bd01fe0d; no voters:
I20260807 04:14:28.751786 5505 leader_election.cc:288] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:28.751955 5511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:28.752797 5511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [term 1 LEADER]: Becoming Leader. State: Replica: 1b212360f88249d497fa7877bd01fe0d, State: Running, Role: LEADER
I20260807 04:14:28.753386 5511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b212360f88249d497fa7877bd01fe0d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34961 } }
I20260807 04:14:28.754179 5505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:28.756479 5512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b212360f88249d497fa7877bd01fe0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b212360f88249d497fa7877bd01fe0d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34961 } } }
I20260807 04:14:28.757242 5512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:28.756738 5513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b212360f88249d497fa7877bd01fe0d. Latest consensus state: current_term: 1 leader_uuid: "1b212360f88249d497fa7877bd01fe0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b212360f88249d497fa7877bd01fe0d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34961 } } }
I20260807 04:14:28.760725 5513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:28.761713 5518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:28.768101 5518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:28.777292 5518 catalog_manager.cc:1347] Generated new cluster ID: 9d9699262a3a455abe9b3991581058ba
I20260807 04:14:28.777433 5518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:28.785851 5518 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:28.787635 5518 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:28.795336 5518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d: Generated new TSK 0
I20260807 04:14:28.796326 5518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:28.807773 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34961
--builtin_ntp_servers=127.18.193.20:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:14:29.161206 5532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:29.161499 5532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:29.161586 5532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:29.169198 5532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:29.169306 5532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:29.169414 5532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:14:29.178725 5532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:29.180001 5532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:29.181840 5532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:29.195165 5537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:29.195243 5538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:29.195209 5540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:29.196139 5532 server_base.cc:1034] running on GCE node
I20260807 04:14:29.196825 5532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:29.197916 5532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:29.199184 5532 hybrid_clock.cc:648] HybridClock initialized: now 1786076069199128 us; error 48 us; skew 500 ppm
I20260807 04:14:29.199559 5532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:29.202044 5532 webserver.cc:460] Webserver started at http://127.18.193.1:33305/ using document root <none> and password file <none>
I20260807 04:14:29.202757 5532 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:29.202878 5532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:29.203222 5532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:29.206223 5532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "f3dd952b960e48cf953714e5f09cfef2"
format_stamp: "Formatted at 2026-08-07 04:14:29 on dist-test-slave-8n49"
server_key: "1c0abbea60b7acb7de2a5ea6b12adfac"
server_key_iv: "9b455af1f1ef287707aa03d06b09c359"
server_key_version: "encryptionkey@0"
I20260807 04:14:29.207042 5532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "f3dd952b960e48cf953714e5f09cfef2"
format_stamp: "Formatted at 2026-08-07 04:14:29 on dist-test-slave-8n49"
server_key: "1c0abbea60b7acb7de2a5ea6b12adfac"
server_key_iv: "9b455af1f1ef287707aa03d06b09c359"
server_key_version: "encryptionkey@0"
I20260807 04:14:29.212342 5532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:14:29.215806 5547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:29.217214 5532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:14:29.217396 5532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "f3dd952b960e48cf953714e5f09cfef2"
format_stamp: "Formatted at 2026-08-07 04:14:29 on dist-test-slave-8n49"
server_key: "1c0abbea60b7acb7de2a5ea6b12adfac"
server_key_iv: "9b455af1f1ef287707aa03d06b09c359"
server_key_version: "encryptionkey@0"
I20260807 04:14:29.217633 5532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:29.245869 5532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:29.246773 5532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:29.247099 5532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:29.248126 5532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:29.249752 5532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:29.249886 5532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:29.250018 5532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:29.250113 5532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:29.284952 5532 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:42099
I20260807 04:14:29.285046 5665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:42099 every 8 connection(s)
I20260807 04:14:29.286365 5532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:14:29.289822 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 5532
I20260807 04:14:29.290045 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:14:29.290540 19204 external_mini_cluster.cc:1428] Setting key 362091c04a9d869df400748c9b00f586
I20260807 04:14:29.293344 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34961
--builtin_ntp_servers=127.18.193.20:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:14:29.301297 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:29.289101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:46063 (local address 127.18.193.62:34961)
0807 04:14:29.289640 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:29.289646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:29.291812 (+ 2166us) server_negotiation.cc:407] Connection header received
0807 04:14:29.293815 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:29.293824 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:29.293916 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:29.294037 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:29.295152 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.296354 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.298018 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.298732 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.298877 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:29.300232 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:29.300241 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:29.300249 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:29.300276 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:29.300329 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:29.300348 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:29.300458 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:29.300728 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 04:14:29.300847 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":253,"threads_started":1}
I20260807 04:14:29.302098 5669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:29.289757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34961 (local address 127.18.193.1:46063)
0807 04:14:29.291641 (+ 1884us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:29.291696 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:29.293521 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:29.294254 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:29.294263 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:29.294350 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:29.294994 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:29.295017 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.296520 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.296525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:29.297894 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:29.297907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.298965 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.298970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:29.299169 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:29.299896 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:29.299899 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:29.299941 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:29.300018 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:29.300447 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:29.300456 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:29.300462 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:14:29.300690 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 04:14:29.300989 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1336,"thread_start_us":1118,"threads_started":1}
I20260807 04:14:29.303704 5666 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34961
I20260807 04:14:29.304018 5666 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:29.304911 5666 heartbeater.cc:511] Master 127.18.193.62:34961 requested a full tablet report, sending...
I20260807 04:14:29.307389 5468 ts_manager.cc:194] Registered new tserver with Master: f3dd952b960e48cf953714e5f09cfef2 (127.18.193.1:42099)
I20260807 04:14:29.309530 5468 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.1:46063
W20260807 04:14:29.639684 5671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:29.639994 5671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:29.640151 5671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:29.647509 5671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:29.647626 5671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:29.647814 5671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:14:29.657717 5671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:29.659137 5671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:29.660998 5671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:29.675129 5678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:29.675259 5680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:29.675254 5677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:29.678299 5671 server_base.cc:1034] running on GCE node
I20260807 04:14:29.679069 5671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:29.680130 5671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:29.681409 5671 hybrid_clock.cc:648] HybridClock initialized: now 1786076069681327 us; error 59 us; skew 500 ppm
I20260807 04:14:29.681807 5671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:29.684257 5671 webserver.cc:460] Webserver started at http://127.18.193.2:41157/ using document root <none> and password file <none>
I20260807 04:14:29.685045 5671 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:29.685182 5671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:29.685537 5671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:29.688737 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "174ce57000294001b75a8e8b06eada18"
format_stamp: "Formatted at 2026-08-07 04:14:29 on dist-test-slave-8n49"
server_key: "f7ac5909cc1b8fef51809e41e7a5b6cb"
server_key_iv: "22b1a593a6bb0cb7fa786463072a1e1d"
server_key_version: "encryptionkey@0"
I20260807 04:14:29.689611 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "174ce57000294001b75a8e8b06eada18"
format_stamp: "Formatted at 2026-08-07 04:14:29 on dist-test-slave-8n49"
server_key: "f7ac5909cc1b8fef51809e41e7a5b6cb"
server_key_iv: "22b1a593a6bb0cb7fa786463072a1e1d"
server_key_version: "encryptionkey@0"
I20260807 04:14:29.695045 5671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:14:29.698618 5686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:29.700039 5671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:14:29.700251 5671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "174ce57000294001b75a8e8b06eada18"
format_stamp: "Formatted at 2026-08-07 04:14:29 on dist-test-slave-8n49"
server_key: "f7ac5909cc1b8fef51809e41e7a5b6cb"
server_key_iv: "22b1a593a6bb0cb7fa786463072a1e1d"
server_key_version: "encryptionkey@0"
I20260807 04:14:29.700487 5671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:29.728703 5671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:29.729713 5671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:29.730059 5671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:29.731223 5671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:29.732878 5671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:29.732977 5671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:29.733060 5671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:29.733119 5671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:29.768293 5671 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:44641
I20260807 04:14:29.768399 5802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:44641 every 8 connection(s)
I20260807 04:14:29.769657 5671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:14:29.777360 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 5671
I20260807 04:14:29.777562 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:14:29.778172 19204 external_mini_cluster.cc:1428] Setting key dd867323e631a5c57baab46bcd8f9ce1
I20260807 04:14:29.781569 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34961
--builtin_ntp_servers=127.18.193.20:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:14:29.783476 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:29.772211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:50365 (local address 127.18.193.62:34961)
0807 04:14:29.772440 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:29.772447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:29.773896 (+ 1449us) server_negotiation.cc:407] Connection header received
0807 04:14:29.775690 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:29.775695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:29.775764 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:29.775859 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:29.776987 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.778143 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.780414 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.781040 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.781183 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:29.782515 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:29.782524 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:29.782532 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:29.782558 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:29.782602 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:29.782621 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:29.782720 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:29.782932 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 04:14:29.783049 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:14:29.784314 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:29.772858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34961 (local address 127.18.193.2:50365)
0807 04:14:29.773740 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:29.773781 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:29.775402 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:29.776065 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:29.776073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:29.776163 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:29.776788 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:29.776806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.778341 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.778346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:29.780210 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:29.780228 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:29.781254 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:29.781259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:29.781456 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:29.782175 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:29.782177 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:29.782219 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:29.782307 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:29.782724 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:29.782733 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:29.782738 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:14:29.782883 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 04:14:29.783197 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":284,"threads_started":1}
I20260807 04:14:29.785925 5803 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34961
I20260807 04:14:29.786309 5803 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:29.787376 5803 heartbeater.cc:511] Master 127.18.193.62:34961 requested a full tablet report, sending...
I20260807 04:14:29.789155 5468 ts_manager.cc:194] Registered new tserver with Master: 174ce57000294001b75a8e8b06eada18 (127.18.193.2:44641)
I20260807 04:14:29.790117 5468 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.2:50365
W20260807 04:14:30.122988 5806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:30.123261 5806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:30.123435 5806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:30.130708 5806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:30.130813 5806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:30.131032 5806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:14:30.140506 5806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:30.141870 5806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:30.143725 5806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:30.157218 5815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:30.157286 5812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:30.157183 5813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:30.158428 5806 server_base.cc:1034] running on GCE node
I20260807 04:14:30.159070 5806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:30.160048 5806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:30.161285 5806 hybrid_clock.cc:648] HybridClock initialized: now 1786076070161236 us; error 58 us; skew 500 ppm
I20260807 04:14:30.161620 5806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:30.163973 5806 webserver.cc:460] Webserver started at http://127.18.193.3:36119/ using document root <none> and password file <none>
I20260807 04:14:30.164757 5806 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:30.164894 5806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:30.165234 5806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:30.168170 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "ddef8e08b77e4225ab065f545a62d164"
format_stamp: "Formatted at 2026-08-07 04:14:30 on dist-test-slave-8n49"
server_key: "4f643f0942eb21042a2900bee77f387d"
server_key_iv: "b3cdb7014134222af2f15879aa53630b"
server_key_version: "encryptionkey@0"
I20260807 04:14:30.169059 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "ddef8e08b77e4225ab065f545a62d164"
format_stamp: "Formatted at 2026-08-07 04:14:30 on dist-test-slave-8n49"
server_key: "4f643f0942eb21042a2900bee77f387d"
server_key_iv: "b3cdb7014134222af2f15879aa53630b"
server_key_version: "encryptionkey@0"
I20260807 04:14:30.174332 5806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:14:30.177838 5822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:30.179363 5806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:14:30.179571 5806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "ddef8e08b77e4225ab065f545a62d164"
format_stamp: "Formatted at 2026-08-07 04:14:30 on dist-test-slave-8n49"
server_key: "4f643f0942eb21042a2900bee77f387d"
server_key_iv: "b3cdb7014134222af2f15879aa53630b"
server_key_version: "encryptionkey@0"
I20260807 04:14:30.179828 5806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:30.206045 5806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:30.206982 5806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:30.207362 5806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:30.208467 5806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:30.210052 5806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:30.210178 5806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:30.210321 5806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:30.210409 5806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:30.244473 5806 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33561
I20260807 04:14:30.244714 5941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33561 every 8 connection(s)
I20260807 04:14:30.245841 5806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:14:30.254415 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 5806
I20260807 04:14:30.254629 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:14:30.255126 19204 external_mini_cluster.cc:1428] Setting key 654e152368c10b2e00032a94cd551257
I20260807 04:14:30.259009 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.248624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:55375 (local address 127.18.193.62:34961)
0807 04:14:30.248809 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.248814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.250128 (+ 1314us) server_negotiation.cc:407] Connection header received
0807 04:14:30.251918 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.251923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.252040 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.252142 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:30.253472 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.254408 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.256250 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.256730 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.256870 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.258312 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:30.258317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:30.258320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:30.258337 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:30.258368 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:30.258380 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:30.258453 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.258666 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.258744 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:14:30.259984 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.249110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34961 (local address 127.18.193.3:55375)
0807 04:14:30.249972 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.250011 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:30.251652 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:30.252456 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:30.252465 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:30.252642 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:30.253309 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.253328 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.254614 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.254638 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.256097 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.256110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.256965 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.256973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.257210 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.258000 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:30.258004 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:30.258053 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:30.258136 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:30.258474 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:30.258482 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:30.258488 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:14:30.258632 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 04:14:30.258918 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":260,"threads_started":1}
I20260807 04:14:30.261350 5942 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34961
I20260807 04:14:30.261674 5942 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:30.262394 5942 heartbeater.cc:511] Master 127.18.193.62:34961 requested a full tablet report, sending...
I20260807 04:14:30.263932 5468 ts_manager.cc:194] Registered new tserver with Master: ddef8e08b77e4225ab065f545a62d164 (127.18.193.3:33561)
I20260807 04:14:30.264779 5468 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.3:55375
I20260807 04:14:30.270447 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:14:30.280704 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.274728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.18.193.62:34961)
0807 04:14:30.274931 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.274939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.275318 (+ 379us) server_negotiation.cc:407] Connection header received
0807 04:14:30.275526 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.275531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.275606 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.275708 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:30.276213 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.277023 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.278373 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.278775 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.278904 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.279927 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:30.279933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:30.279937 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:30.279963 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:30.279993 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:30.280005 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:30.280099 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.280254 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.280343 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:14:30.284483 5468 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:14:30.293766 5468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:14:30.313503 5666 heartbeater.cc:503] Master 127.18.193.62:34961 was elected leader, sending a full tablet report...
I20260807 04:14:30.316648 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.305787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56004)
0807 04:14:30.306601 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.306658 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:30.306855 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:30.308158 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:30.308166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:30.308200 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:30.308675 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.308690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.311076 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.311084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.313378 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.313398 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.315529 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.315536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.315770 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.315829 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 04:14:30.315997 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 04:14:30.316199 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":1,"thread_start_us":312,"threads_started":1}
I20260807 04:14:30.316629 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.305913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.18.193.1:42099)
0807 04:14:30.307485 (+ 1572us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.307495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.307530 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:14:30.307634 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.307643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.307899 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.308936 (+ 1037us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:30.309239 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.310920 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.313690 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.315324 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.315504 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.315809 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.316038 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.316216 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1393,"thread_start_us":1286,"threads_started":1}
I20260807 04:14:30.317535 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.306219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33561 (local address 127.0.0.1:58772)
0807 04:14:30.307764 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.307781 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:30.307886 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:30.309487 (+ 1601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:30.309491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:30.309514 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:30.309896 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.309912 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.312081 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.312086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.314893 (+ 2807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.314904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.316720 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.316727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.316912 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.316958 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 04:14:30.317022 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 04:14:30.317152 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"mutex_wait_us":167,"thread_start_us":1267,"threads_started":1}
I20260807 04:14:30.318186 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.307477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.18.193.2:44641)
0807 04:14:30.308381 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.308389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.309410 (+ 1021us) server_negotiation.cc:407] Connection header received
0807 04:14:30.309540 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.309550 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.309765 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.309951 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:30.310661 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.312130 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.314914 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.316406 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.316592 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.316829 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.317628 (+ 799us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.317803 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":604,"threads_started":1}
I20260807 04:14:30.318665 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.306938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44641 (local address 127.0.0.1:37212)
0807 04:14:30.309162 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.309180 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:30.309288 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:30.310029 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:30.310033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:30.310062 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:30.310478 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.310502 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.312299 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.312306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.314668 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.314683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.316979 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.316986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.317163 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.317191 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 04:14:30.317269 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 04:14:30.317380 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2104,"mutex_wait_us":29,"thread_start_us":1982,"threads_started":1}
I20260807 04:14:30.319092 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.306877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.18.193.3:33561)
0807 04:14:30.308926 (+ 2049us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.308935 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.308969 (+ 34us) server_negotiation.cc:407] Connection header received
0807 04:14:30.309065 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.309073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.309292 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.310129 (+ 837us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:30.310365 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.311934 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.315092 (+ 3158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.316443 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.318128 (+ 1685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.318428 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.318495 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.318692 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1864,"thread_start_us":1708,"threads_started":1}
I20260807 04:14:30.321909 5597 tablet_service.cc:1467] Processing CreateTablet for tablet 09f912e9d20b4e9e844f0930d6255017 (DEFAULT_TABLE table=test-table [id=baf67096b33a4afaa85f42d9821e894d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:30.322245 5874 tablet_service.cc:1467] Processing CreateTablet for tablet 09f912e9d20b4e9e844f0930d6255017 (DEFAULT_TABLE table=test-table [id=baf67096b33a4afaa85f42d9821e894d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:30.322589 5735 tablet_service.cc:1467] Processing CreateTablet for tablet 09f912e9d20b4e9e844f0930d6255017 (DEFAULT_TABLE table=test-table [id=baf67096b33a4afaa85f42d9821e894d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:30.323555 5874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f912e9d20b4e9e844f0930d6255017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:30.323581 5597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f912e9d20b4e9e844f0930d6255017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:30.323932 5735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f912e9d20b4e9e844f0930d6255017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:30.335019 5960 tablet_bootstrap.cc:492] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Bootstrap starting.
I20260807 04:14:30.338274 5960 tablet_bootstrap.cc:654] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:30.339435 5960 log.cc:824] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:30.339811 5961 tablet_bootstrap.cc:492] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Bootstrap starting.
I20260807 04:14:30.340752 5962 tablet_bootstrap.cc:492] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: Bootstrap starting.
I20260807 04:14:30.345712 5961 tablet_bootstrap.cc:654] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:30.346908 5961 log.cc:824] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:30.348088 5962 tablet_bootstrap.cc:654] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:30.348934 5960 tablet_bootstrap.cc:492] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: No bootstrap required, opened a new log
I20260807 04:14:30.349197 5962 log.cc:824] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:30.349243 5960 ts_tablet_manager.cc:1397] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260807 04:14:30.350088 5961 tablet_bootstrap.cc:492] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: No bootstrap required, opened a new log
I20260807 04:14:30.351280 5961 ts_tablet_manager.cc:1397] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 04:14:30.353200 5962 tablet_bootstrap.cc:492] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: No bootstrap required, opened a new log
I20260807 04:14:30.353474 5962 ts_tablet_manager.cc:1397] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260807 04:14:30.360203 5960 raft_consensus.cc:348] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.360364 5962 raft_consensus.cc:348] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.360855 5962 raft_consensus.cc:374] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:30.360888 5960 raft_consensus.cc:374] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:30.361024 5962 raft_consensus.cc:729] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3dd952b960e48cf953714e5f09cfef2, State: Initialized, Role: FOLLOWER
I20260807 04:14:30.361060 5960 raft_consensus.cc:729] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddef8e08b77e4225ab065f545a62d164, State: Initialized, Role: FOLLOWER
I20260807 04:14:30.361909 5962 consensus_queue.cc:260] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.361910 5960 consensus_queue.cc:260] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.361822 5961 raft_consensus.cc:348] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.362254 5961 raft_consensus.cc:374] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:30.362360 5961 raft_consensus.cc:729] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 174ce57000294001b75a8e8b06eada18, State: Initialized, Role: FOLLOWER
I20260807 04:14:30.363214 5942 heartbeater.cc:503] Master 127.18.193.62:34961 was elected leader, sending a full tablet report...
I20260807 04:14:30.364001 5960 ts_tablet_manager.cc:1428] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 04:14:30.364362 5962 ts_tablet_manager.cc:1428] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260807 04:14:30.364465 5961 consensus_queue.cc:260] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.366783 5803 heartbeater.cc:503] Master 127.18.193.62:34961 was elected leader, sending a full tablet report...
I20260807 04:14:30.367411 5961 ts_tablet_manager.cc:1428] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
W20260807 04:14:30.497764 5943 tablet.cc:2369] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:14:30.521935 5804 tablet.cc:2369] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:14:30.539044 5667 tablet.cc:2369] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:30.565646 5966 raft_consensus.cc:482] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:30.566005 5966 raft_consensus.cc:504] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.568003 5966 leader_election.cc:288] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 174ce57000294001b75a8e8b06eada18 (127.18.193.2:44641), ddef8e08b77e4225ab065f545a62d164 (127.18.193.3:33561)
I20260807 04:14:30.575264 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.568588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:40973 (local address 127.18.193.2:44641)
0807 04:14:30.568829 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.568837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.569709 (+ 872us) server_negotiation.cc:407] Connection header received
0807 04:14:30.569940 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.569945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.570016 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.570113 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:30.570731 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.571488 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.573633 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.574364 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.574544 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.574622 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.574871 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.575012 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 04:14:30.575620 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.568586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44641 (local address 127.18.193.1:40973)
0807 04:14:30.569584 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.569601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:30.569790 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:30.570145 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:30.570150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:30.570175 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:30.570580 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.570589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.571648 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.571656 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.573431 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.573444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.574549 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.574556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.574710 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.574734 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:14:30.574835 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 04:14:30.574962 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":89,"spinlock_wait_cycles":3456,"thread_start_us":579,"threads_started":1}
I20260807 04:14:30.576028 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.568720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:57389 (local address 127.18.193.3:33561)
0807 04:14:30.568965 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.568974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.569784 (+ 810us) server_negotiation.cc:407] Connection header received
0807 04:14:30.570018 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.570022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.570086 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.570192 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:30.570960 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.571895 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.574078 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.575129 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.575335 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.575400 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.575674 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.575800 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 04:14:30.576138 5970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.568706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33561 (local address 127.18.193.1:57389)
0807 04:14:30.569658 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.569674 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:30.569857 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:30.570208 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:30.570213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:30.570251 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:30.570725 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.570737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.572045 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.572055 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.573883 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:30.573896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.575356 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.575364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:30.575514 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.575536 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 04:14:30.575650 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 04:14:30.575771 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":3,"thread_start_us":588,"threads_started":1}
I20260807 04:14:30.576885 5756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f912e9d20b4e9e844f0930d6255017" candidate_uuid: "f3dd952b960e48cf953714e5f09cfef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "174ce57000294001b75a8e8b06eada18" is_pre_election: true
I20260807 04:14:30.577487 5756 raft_consensus.cc:2454] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3dd952b960e48cf953714e5f09cfef2 in term 0.
I20260807 04:14:30.577502 5894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f912e9d20b4e9e844f0930d6255017" candidate_uuid: "f3dd952b960e48cf953714e5f09cfef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddef8e08b77e4225ab065f545a62d164" is_pre_election: true
I20260807 04:14:30.578051 5894 raft_consensus.cc:2454] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3dd952b960e48cf953714e5f09cfef2 in term 0.
I20260807 04:14:30.578154 5551 leader_election.cc:302] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 174ce57000294001b75a8e8b06eada18, f3dd952b960e48cf953714e5f09cfef2; no voters:
I20260807 04:14:30.578615 5966 raft_consensus.cc:2781] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:14:30.578747 5966 raft_consensus.cc:482] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:14:30.578889 5966 raft_consensus.cc:3037] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:30.581135 5966 raft_consensus.cc:504] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.581805 5966 leader_election.cc:288] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [CANDIDATE]: Term 1 election: Requested vote from peers 174ce57000294001b75a8e8b06eada18 (127.18.193.2:44641), ddef8e08b77e4225ab065f545a62d164 (127.18.193.3:33561)
I20260807 04:14:30.582217 5756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f912e9d20b4e9e844f0930d6255017" candidate_uuid: "f3dd952b960e48cf953714e5f09cfef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "174ce57000294001b75a8e8b06eada18"
I20260807 04:14:30.582273 5894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f912e9d20b4e9e844f0930d6255017" candidate_uuid: "f3dd952b960e48cf953714e5f09cfef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddef8e08b77e4225ab065f545a62d164"
I20260807 04:14:30.582427 5756 raft_consensus.cc:3037] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:30.582468 5894 raft_consensus.cc:3037] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:30.584867 5894 raft_consensus.cc:2454] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3dd952b960e48cf953714e5f09cfef2 in term 1.
I20260807 04:14:30.584877 5756 raft_consensus.cc:2454] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3dd952b960e48cf953714e5f09cfef2 in term 1.
I20260807 04:14:30.585282 5549 leader_election.cc:302] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ddef8e08b77e4225ab065f545a62d164, f3dd952b960e48cf953714e5f09cfef2; no voters:
I20260807 04:14:30.585593 5966 raft_consensus.cc:2781] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:30.586053 5966 raft_consensus.cc:686] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 1 LEADER]: Becoming Leader. State: Replica: f3dd952b960e48cf953714e5f09cfef2, State: Running, Role: LEADER
I20260807 04:14:30.586519 5966 consensus_queue.cc:237] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } }
I20260807 04:14:30.591264 5466 catalog_manager.cc:5495] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3dd952b960e48cf953714e5f09cfef2 (127.18.193.1). New cstate: current_term: 1 leader_uuid: "f3dd952b960e48cf953714e5f09cfef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3dd952b960e48cf953714e5f09cfef2" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 } health_report { overall_health: UNKNOWN } } }
I20260807 04:14:30.618331 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.612465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.18.193.1:42099)
0807 04:14:30.612729 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.612739 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.612810 (+ 71us) server_negotiation.cc:407] Connection header received
0807 04:14:30.613072 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.613076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.613170 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.613302 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:14:30.613630 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.614315 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.615924 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.616421 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.616537 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.617136 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:14:30.617706 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:14:30.617828 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.617923 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.617993 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:14:30.626935 5756 raft_consensus.cc:1261] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 1 FOLLOWER]: Refusing update from remote peer f3dd952b960e48cf953714e5f09cfef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:30.626919 5894 raft_consensus.cc:1261] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 1 FOLLOWER]: Refusing update from remote peer f3dd952b960e48cf953714e5f09cfef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:30.628046 5971 consensus_queue.cc:1035] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddef8e08b77e4225ab065f545a62d164" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:30.628460 5966 consensus_queue.cc:1035] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "174ce57000294001b75a8e8b06eada18" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:30.640316 5974 mvcc.cc:205] Tried to move back new op lower bound from 7315767585277030400 to 7315767585125912576. Current Snapshot: MvccSnapshot[applied={T|T < 7315767585277030400}]
I20260807 04:14:30.641914 5976 mvcc.cc:205] Tried to move back new op lower bound from 7315767585277030400 to 7315767585125912576. Current Snapshot: MvccSnapshot[applied={T|T < 7315767585277030400}]
I20260807 04:14:30.642223 5975 mvcc.cc:205] Tried to move back new op lower bound from 7315767585277030400 to 7315767585125912576. Current Snapshot: MvccSnapshot[applied={T|T < 7315767585277030400}]
I20260807 04:14:30.653683 5466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:30.654095 5466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:30.658004 5466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b212360f88249d497fa7877bd01fe0d: Sending DeleteTablet for 3 replicas of tablet 09f912e9d20b4e9e844f0930d6255017
I20260807 04:14:30.659245 5735 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f912e9d20b4e9e844f0930d6255017 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:30 UTC) from {username='slave'} at 127.0.0.1:37212
I20260807 04:14:30.659252 5597 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f912e9d20b4e9e844f0930d6255017 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:30 UTC) from {username='slave'} at 127.0.0.1:56004
I20260807 04:14:30.659307 5874 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f912e9d20b4e9e844f0930d6255017 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:30 UTC) from {username='slave'} at 127.0.0.1:58772
I20260807 04:14:30.661152 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 5532
I20260807 04:14:30.672621 5987 tablet_replica.cc:330] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: stopping tablet replica
I20260807 04:14:30.673255 5987 raft_consensus.cc:2229] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:14:30.674059 5987 raft_consensus.cc:2258] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:30.676906 5987 ts_tablet_manager.cc:1905] T 09f912e9d20b4e9e844f0930d6255017 P f3dd952b960e48cf953714e5f09cfef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:30.677551 5986 tablet_replica.cc:330] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: stopping tablet replica
I20260807 04:14:30.678215 5986 raft_consensus.cc:2229] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:30.678294 5988 tablet_replica.cc:330] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: stopping tablet replica
I20260807 04:14:30.678673 5988 raft_consensus.cc:2229] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:30.679246 5988 raft_consensus.cc:2258] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:30.679382 5986 raft_consensus.cc:2258] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:30.681739 5986 ts_tablet_manager.cc:1905] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:30.682142 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.662817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56022 (local address 127.18.193.1:42099)
0807 04:14:30.663092 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.663101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.668031 (+ 4930us) server_negotiation.cc:407] Connection header received
0807 04:14:30.668243 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.668252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.668384 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.668501 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:30.670671 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.672271 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.675994 (+ 3723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.676655 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.676762 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.680292 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:30.680305 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:30.680327 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:30.681185 (+ 858us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:30.681245 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:30.681277 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:30.681380 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.681627 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.681747 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 04:14:30.683310 5661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:30.685695 5988 ts_tablet_manager.cc:1905] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:30.690402 5986 ts_tablet_manager.cc:1918] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:30.690575 5986 log.cc:1196] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/09f912e9d20b4e9e844f0930d6255017
I20260807 04:14:30.691179 5986 ts_tablet_manager.cc:1939] T 09f912e9d20b4e9e844f0930d6255017 P 174ce57000294001b75a8e8b06eada18: Deleting consensus metadata
I20260807 04:14:30.691849 5988 ts_tablet_manager.cc:1918] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:30.692030 5988 log.cc:1196] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/09f912e9d20b4e9e844f0930d6255017
I20260807 04:14:30.692603 5988 ts_tablet_manager.cc:1939] T 09f912e9d20b4e9e844f0930d6255017 P ddef8e08b77e4225ab065f545a62d164: Deleting consensus metadata
I20260807 04:14:30.693364 5454 catalog_manager.cc:4841] TS 174ce57000294001b75a8e8b06eada18 (127.18.193.2:44641): tablet 09f912e9d20b4e9e844f0930d6255017 (table test-table [id=baf67096b33a4afaa85f42d9821e894d]) successfully deleted
I20260807 04:14:30.694198 5452 catalog_manager.cc:4841] TS ddef8e08b77e4225ab065f545a62d164 (127.18.193.3:33561): tablet 09f912e9d20b4e9e844f0930d6255017 (table test-table [id=baf67096b33a4afaa85f42d9821e894d]) successfully deleted
I20260807 04:14:30.782899 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 5532
W20260807 04:14:30.805212 5454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.193.1:42099 (error 108)
I20260807 04:14:30.805413 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 5671
I20260807 04:14:30.807067 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.806574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56030)
0807 04:14:30.806756 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.806876 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 04:14:30.807327 5454 catalog_manager.cc:4584] TS f3dd952b960e48cf953714e5f09cfef2 (127.18.193.1:42099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09f912e9d20b4e9e844f0930d6255017: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
I20260807 04:14:30.812817 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.806299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.18.193.2:44641)
0807 04:14:30.806508 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.806514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.806534 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:14:30.806771 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.806777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.806879 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.806963 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:30.807484 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.808290 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.809635 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.810043 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.810145 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.811104 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:30.811112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:30.811125 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:30.811748 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:30.811795 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:30.811815 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:30.811920 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.812091 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.812176 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:14:30.813491 5798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:30.860965 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.860373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56038)
0807 04:14:30.860648 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.860756 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 04:14:30.862103 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.861665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56048)
0807 04:14:30.861823 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.861919 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 04:14:30.908180 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 5671
I20260807 04:14:30.922719 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.922232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56056)
0807 04:14:30.922451 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.922535 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 04:14:30.924067 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.923467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56060)
0807 04:14:30.923686 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:30.923795 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 04:14:30.928732 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 5806
I20260807 04:14:30.935461 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:30.929460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.18.193.3:33561)
0807 04:14:30.929617 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:30.929621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:30.929734 (+ 113us) server_negotiation.cc:407] Connection header received
0807 04:14:30.929869 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:30.929873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:30.929963 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:30.930058 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:30.930508 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.931259 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.932643 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:30.933064 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:30.933166 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:30.934210 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:30.934220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:30.934235 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:30.934814 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:30.934862 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:30.934883 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:30.934983 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:30.935084 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 04:14:30.935171 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 04:14:30.936228 5937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:31.005010 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:31.004448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56070)
0807 04:14:31.004726 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:31.004828 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 04:14:31.006119 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:31.005739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42099 (local address 127.0.0.1:56080)
0807 04:14:31.005946 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:31.006013 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 04:14:31.025738 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 5806
I20260807 04:14:31.044420 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 5434
I20260807 04:14:31.045655 5500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:31.158499 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 5434
2026-08-07T04:14:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2991 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:31 dist-test-slave-8n49 krb5kdc[5998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:31 dist-test-slave-8n49 krb5kdc[5998](info): set up 2 sockets
Aug 07 04:14:31 dist-test-slave-8n49 krb5kdc[5998](info): commencing operation
krb5kdc: starting...
W20260807 04:14:32.335228 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:32 dist-test-slave-8n49 krb5kdc[5998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:14:32.636524 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34705
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:34705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:32.977360 6012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:32.977655 6012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:32.977845 6012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:32.985257 6012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:32.985360 6012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:32.985466 6012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:32.985586 6012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:32.985644 6012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:32.995803 6012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:34705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:32.997241 6012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:32.998577 6012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:33.011094 6018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:33.011125 6020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:33.011122 6017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:33.012089 6012 server_base.cc:1034] running on GCE node
I20260807 04:14:33.012976 6012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:33.014429 6012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:33.015679 6012 hybrid_clock.cc:648] HybridClock initialized: now 1786076073015635 us; error 54 us; skew 500 ppm
I20260807 04:14:33.016018 6012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:33 dist-test-slave-8n49 krb5kdc[5998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076073, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:33.019232 6012 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:14:33.020851 6012 webserver.cc:460] Webserver started at http://127.18.193.62:46611/ using document root <none> and password file <none>
I20260807 04:14:33.021552 6012 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:33.021694 6012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:33.021997 6012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:33.025080 6012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "9737add050e6421988ef011d1d2d5b6f"
format_stamp: "Formatted at 2026-08-07 04:14:33 on dist-test-slave-8n49"
server_key: "acc2df79936766de1d33458f180d7bc2"
server_key_iv: "407811d0df8d1ad9712fbc6ab7068ba4"
server_key_version: "encryptionkey@0"
I20260807 04:14:33.025840 6012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "9737add050e6421988ef011d1d2d5b6f"
format_stamp: "Formatted at 2026-08-07 04:14:33 on dist-test-slave-8n49"
server_key: "acc2df79936766de1d33458f180d7bc2"
server_key_iv: "407811d0df8d1ad9712fbc6ab7068ba4"
server_key_version: "encryptionkey@0"
I20260807 04:14:33.031137 6012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 04:14:33.034821 6027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:33.036165 6012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:14:33.036352 6012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "9737add050e6421988ef011d1d2d5b6f"
format_stamp: "Formatted at 2026-08-07 04:14:33 on dist-test-slave-8n49"
server_key: "acc2df79936766de1d33458f180d7bc2"
server_key_iv: "407811d0df8d1ad9712fbc6ab7068ba4"
server_key_version: "encryptionkey@0"
I20260807 04:14:33.036597 6012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:33.053933 6012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:33.058503 6012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:33.058820 6012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:33.078356 6012 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:34705
I20260807 04:14:33.078399 6084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:34705 every 8 connection(s)
I20260807 04:14:33.079773 6012 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:33.083640 6085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:33.086107 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 6012
I20260807 04:14:33.086323 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:33.086897 19204 external_mini_cluster.cc:1428] Setting key 86e8f553b94d4cf437196fa5322751e8
I20260807 04:14:33.096901 6085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f: Bootstrap starting.
I20260807 04:14:33.100967 6085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:33.102420 6085 log.cc:824] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:14:33 dist-test-slave-8n49 krb5kdc[5998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:33.105792 6085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f: No bootstrap required, opened a new log
I20260807 04:14:33.112207 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:33.089068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.18.193.62:34705)
0807 04:14:33.090076 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:33.090089 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:33.090132 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:14:33.091852 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:33.091875 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:33.092277 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:33.092711 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:33.094521 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:33.095969 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:33.097966 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:33.098764 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:33.098932 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:33.104515 (+ 5583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:33.104525 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:33.104540 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:33.104651 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:33.107876 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:33.108673 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:33.108683 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:33.108690 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:33.108860 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:33.109313 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:33.109322 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:33.109324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:33.109873 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:33.110242 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:33.110758 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 04:14:33.111191 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":487,"threads_started":1}
I20260807 04:14:33.113897 6085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9737add050e6421988ef011d1d2d5b6f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34705 } }
I20260807 04:14:33.114285 6085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:33.114418 6085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9737add050e6421988ef011d1d2d5b6f, State: Initialized, Role: FOLLOWER
I20260807 04:14:33.114974 6085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9737add050e6421988ef011d1d2d5b6f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34705 } }
I20260807 04:14:33.115262 6085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:33.115419 6085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:33.115554 6085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:33.118127 6085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9737add050e6421988ef011d1d2d5b6f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34705 } }
I20260807 04:14:33.118755 6085 leader_election.cc:302] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9737add050e6421988ef011d1d2d5b6f; no voters:
I20260807 04:14:33.119506 6085 leader_election.cc:288] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:33.119642 6090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:33.121055 6090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [term 1 LEADER]: Becoming Leader. State: Replica: 9737add050e6421988ef011d1d2d5b6f, State: Running, Role: LEADER
I20260807 04:14:33.121591 6090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9737add050e6421988ef011d1d2d5b6f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34705 } }
I20260807 04:14:33.122319 6085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:33.124840 6092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9737add050e6421988ef011d1d2d5b6f. Latest consensus state: current_term: 1 leader_uuid: "9737add050e6421988ef011d1d2d5b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9737add050e6421988ef011d1d2d5b6f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34705 } } }
I20260807 04:14:33.125341 6092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:33.124877 6091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9737add050e6421988ef011d1d2d5b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9737add050e6421988ef011d1d2d5b6f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34705 } } }
I20260807 04:14:33.125962 6091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:33.127781 6096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:33.134120 6096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:33.142521 6096 catalog_manager.cc:1347] Generated new cluster ID: 6cfcf42cf73346af8889fef8fece6558
I20260807 04:14:33.142640 6096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:33.160116 6096 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:33.161798 6096 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:33.168222 6096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f: Generated new TSK 0
I20260807 04:14:33.169353 6096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:14:33.253027 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34705
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:33.606115 6112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:33.606448 6112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:33.606621 6112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:33.614287 6112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:33.614447 6112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:33.614621 6112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:14:33.624945 6112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:33.626353 6112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:33.627723 6112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:33.641520 6117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:33.641526 6120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:33.641578 6118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:33.642426 6112 server_base.cc:1034] running on GCE node
I20260807 04:14:33.643088 6112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:33.644076 6112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:33.645278 6112 hybrid_clock.cc:648] HybridClock initialized: now 1786076073645239 us; error 37 us; skew 500 ppm
I20260807 04:14:33.645634 6112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:33 dist-test-slave-8n49 krb5kdc[5998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076073, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:33.653173 6112 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:14:33.654618 6112 webserver.cc:460] Webserver started at http://127.18.193.1:34951/ using document root <none> and password file <none>
I20260807 04:14:33.655272 6112 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:33.655375 6112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:33.655651 6112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:33.658581 6112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "d12d29e9390144b085342c4cb672cdec"
format_stamp: "Formatted at 2026-08-07 04:14:33 on dist-test-slave-8n49"
server_key: "2f626d9be40820156c6285cf621607b9"
server_key_iv: "ed0ef5ca0a81219ee8b6d9a673ecaa45"
server_key_version: "encryptionkey@0"
I20260807 04:14:33.659292 6112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "d12d29e9390144b085342c4cb672cdec"
format_stamp: "Formatted at 2026-08-07 04:14:33 on dist-test-slave-8n49"
server_key: "2f626d9be40820156c6285cf621607b9"
server_key_iv: "ed0ef5ca0a81219ee8b6d9a673ecaa45"
server_key_version: "encryptionkey@0"
I20260807 04:14:33.664601 6112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:14:33.668128 6128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:33.669549 6112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:33.669734 6112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "d12d29e9390144b085342c4cb672cdec"
format_stamp: "Formatted at 2026-08-07 04:14:33 on dist-test-slave-8n49"
server_key: "2f626d9be40820156c6285cf621607b9"
server_key_iv: "ed0ef5ca0a81219ee8b6d9a673ecaa45"
server_key_version: "encryptionkey@0"
I20260807 04:14:33.669938 6112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:33.688895 6112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:33.693487 6112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:33.693796 6112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:33.694818 6112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:33.696244 6112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:33.696372 6112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:33.696565 6112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:33.696650 6112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:33.731545 6112 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:42083
I20260807 04:14:33.731592 6248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:42083 every 8 connection(s)
I20260807 04:14:33.732908 6112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:14:33.737524 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 6112
I20260807 04:14:33.737739 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:14:33.738328 19204 external_mini_cluster.cc:1428] Setting key 054847b1ce220a3f4648afe5483c2d93
Aug 07 04:14:33 dist-test-slave-8n49 krb5kdc[5998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076073, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:33.755719 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:33.735671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:45001 (local address 127.18.193.62:34705)
0807 04:14:33.736175 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:33.736181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:33.737209 (+ 1028us) server_negotiation.cc:407] Connection header received
0807 04:14:33.739643 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:33.739649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:33.739733 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:33.739849 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:33.741874 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:33.742610 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:33.745125 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:33.745581 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:33.745688 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:33.750033 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:33.750038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:33.750041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:33.750077 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:33.752690 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:33.753428 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:33.753434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:33.753438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:33.753566 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:33.754006 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:33.754011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:33.754015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:33.754283 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:33.754440 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:33.755125 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 04:14:33.755353 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":247,"threads_started":1}
I20260807 04:14:33.756791 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:33.736151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34705 (local address 127.18.193.1:45001)
0807 04:14:33.736998 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:33.737037 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:33.739374 (+ 2337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:33.740140 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:33.740149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:33.740856 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:33.741673 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:33.741702 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:33.742783 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:33.742791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:33.744970 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:33.744988 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:33.745830 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:33.745839 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:33.746184 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:33.746978 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:33.747020 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:33.749770 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:33.752829 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:33.752838 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:33.752856 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:33.753301 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:33.753680 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:33.753685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:33.753689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:33.753883 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:33.754413 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:33.754421 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:33.754758 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 04:14:33.755015 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 04:14:33.755556 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":354,"threads_started":1}
I20260807 04:14:33.758401 6249 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34705
I20260807 04:14:33.758737 6249 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:33.759649 6249 heartbeater.cc:511] Master 127.18.193.62:34705 requested a full tablet report, sending...
I20260807 04:14:33.762161 6044 ts_manager.cc:194] Registered new tserver with Master: d12d29e9390144b085342c4cb672cdec (127.18.193.1:42083)
I20260807 04:14:33.763667 6044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:45001
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:14:33.805557 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34705
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:34.150876 6255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:34.151197 6255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:34.151373 6255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:34.158757 6255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:34.158902 6255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:34.159091 6255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:14:34.168721 6255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:34.169993 6255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:34.171303 6255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:34.184937 6261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:34.185004 6262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:34.184983 6264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:34.185976 6255 server_base.cc:1034] running on GCE node
I20260807 04:14:34.186599 6255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:34.187563 6255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:34.188787 6255 hybrid_clock.cc:648] HybridClock initialized: now 1786076074188743 us; error 60 us; skew 500 ppm
I20260807 04:14:34.189116 6255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:34 dist-test-slave-8n49 krb5kdc[5998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076074, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:34.192181 6255 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:14:34.193742 6255 webserver.cc:460] Webserver started at http://127.18.193.2:34167/ using document root <none> and password file <none>
I20260807 04:14:34.194447 6255 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:34.194608 6255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:34.194936 6255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:34.197984 6255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "364501ac3f8241efa9b0a3060b964861"
format_stamp: "Formatted at 2026-08-07 04:14:34 on dist-test-slave-8n49"
server_key: "f4152310a9daf01a6be908f3cf643658"
server_key_iv: "4a1f096fb2f41ce08b7eaddd007d0c90"
server_key_version: "encryptionkey@0"
I20260807 04:14:34.198761 6255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "364501ac3f8241efa9b0a3060b964861"
format_stamp: "Formatted at 2026-08-07 04:14:34 on dist-test-slave-8n49"
server_key: "f4152310a9daf01a6be908f3cf643658"
server_key_iv: "4a1f096fb2f41ce08b7eaddd007d0c90"
server_key_version: "encryptionkey@0"
I20260807 04:14:34.203943 6255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 04:14:34.207366 6272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:34.208752 6255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:34.208880 6255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "364501ac3f8241efa9b0a3060b964861"
format_stamp: "Formatted at 2026-08-07 04:14:34 on dist-test-slave-8n49"
server_key: "f4152310a9daf01a6be908f3cf643658"
server_key_iv: "4a1f096fb2f41ce08b7eaddd007d0c90"
server_key_version: "encryptionkey@0"
I20260807 04:14:34.209049 6255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:34.223816 6255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:34.227849 6255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:34.228101 6255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:34.229128 6255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:34.230571 6255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:34.230681 6255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:34.230816 6255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:34.230897 6255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:34.263509 6255 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:44733
I20260807 04:14:34.263558 6394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:44733 every 8 connection(s)
I20260807 04:14:34.264991 6255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:14:34.274783 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 6255
I20260807 04:14:34.274998 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:14:34.275511 19204 external_mini_cluster.cc:1428] Setting key de3f093a83f0da3041c322d9e54e1c72
Aug 07 04:14:34 dist-test-slave-8n49 krb5kdc[5998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076074, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:34.287701 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.267631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:50879 (local address 127.18.193.62:34705)
0807 04:14:34.268128 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:34.268135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:34.269253 (+ 1118us) server_negotiation.cc:407] Connection header received
0807 04:14:34.271211 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:34.271216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:34.271291 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:34.271390 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:34.273232 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.274074 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.276157 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.276791 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.276941 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.281249 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:34.281254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:34.281258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:34.281293 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:34.284258 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:34.285140 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:34.285151 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:34.285154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:34.285294 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:34.285791 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:34.285796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:34.285798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:34.286098 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:34.286272 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:34.287000 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 04:14:34.287270 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":237,"threads_started":1}
I20260807 04:14:34.288803 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.268229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34705 (local address 127.18.193.2:50879)
0807 04:14:34.269081 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:34.269132 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:34.270955 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:34.271715 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:34.271723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:34.272288 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:34.273025 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.273045 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.274253 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.274257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.275985 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.276000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.277046 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.277051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.277243 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.278129 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:34.278159 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:34.280991 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:34.284428 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:34.284439 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:34.284456 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:34.284996 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:34.285423 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:34.285429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:34.285433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:34.285650 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:34.286241 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:34.286250 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:34.286619 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 04:14:34.286892 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 04:14:34.287460 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":394,"threads_started":1}
I20260807 04:14:34.290395 6395 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34705
I20260807 04:14:34.290733 6395 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:34.291510 6395 heartbeater.cc:511] Master 127.18.193.62:34705 requested a full tablet report, sending...
I20260807 04:14:34.293182 6044 ts_manager.cc:194] Registered new tserver with Master: 364501ac3f8241efa9b0a3060b964861 (127.18.193.2:44733)
I20260807 04:14:34.294425 6044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:50879
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:14:34.343935 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34705
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:34.682826 6402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:34.683092 6402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:34.683187 6402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:34.690297 6402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:34.690418 6402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:34.690537 6402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:14:34.700238 6402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:34.701737 6402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:34.703050 6402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:34.716507 6412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:34.716478 6409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:34.716536 6410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:34.717286 6402 server_base.cc:1034] running on GCE node
I20260807 04:14:34.717873 6402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:34.718842 6402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:34.720080 6402 hybrid_clock.cc:648] HybridClock initialized: now 1786076074720038 us; error 53 us; skew 500 ppm
I20260807 04:14:34.720403 6402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:34 dist-test-slave-8n49 krb5kdc[5998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076074, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:34.723798 6402 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:14:34.725361 6402 webserver.cc:460] Webserver started at http://127.18.193.3:41307/ using document root <none> and password file <none>
I20260807 04:14:34.726042 6402 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:34.726192 6402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:34.726507 6402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:34.729509 6402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "02cfbe4875c4483ab131a37869c6f582"
format_stamp: "Formatted at 2026-08-07 04:14:34 on dist-test-slave-8n49"
server_key: "b81de7c355ef8bed362a8e4ed7dfcf29"
server_key_iv: "b1e6f1c95fef48578ed292f6f912c00c"
server_key_version: "encryptionkey@0"
I20260807 04:14:34.730461 6402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "02cfbe4875c4483ab131a37869c6f582"
format_stamp: "Formatted at 2026-08-07 04:14:34 on dist-test-slave-8n49"
server_key: "b81de7c355ef8bed362a8e4ed7dfcf29"
server_key_iv: "b1e6f1c95fef48578ed292f6f912c00c"
server_key_version: "encryptionkey@0"
I20260807 04:14:34.737106 6402 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 04:14:34.740691 6421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:34.742085 6402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:14:34.742214 6402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "02cfbe4875c4483ab131a37869c6f582"
format_stamp: "Formatted at 2026-08-07 04:14:34 on dist-test-slave-8n49"
server_key: "b81de7c355ef8bed362a8e4ed7dfcf29"
server_key_iv: "b1e6f1c95fef48578ed292f6f912c00c"
server_key_version: "encryptionkey@0"
I20260807 04:14:34.742424 6402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:34.767673 6249 heartbeater.cc:503] Master 127.18.193.62:34705 was elected leader, sending a full tablet report...
I20260807 04:14:34.771335 6402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:34.776237 6402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:34.776599 6402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:34.777685 6402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:34.779160 6402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:34.779273 6402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:34.779445 6402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:34.779527 6402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:34.816485 6402 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:33497
I20260807 04:14:34.816524 6549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:33497 every 8 connection(s)
I20260807 04:14:34.817909 6402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:14:34.824311 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 6402
I20260807 04:14:34.824615 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:14:34.825192 19204 external_mini_cluster.cc:1428] Setting key 9237cde97fc5a1c71c00a464fdf5e503
Aug 07 04:14:34 dist-test-slave-8n49 krb5kdc[5998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076074, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:34.839335 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.820590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:37581 (local address 127.18.193.62:34705)
0807 04:14:34.821043 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:34.821048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:34.822143 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 04:14:34.824352 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:34.824357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:34.824469 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:34.824629 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:34.826642 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.827780 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.829670 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.830144 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.830256 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.833627 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:34.833632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:34.833635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:34.833672 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:34.836105 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:34.836897 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:34.836909 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:34.836911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:34.837023 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:34.837450 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:34.837455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:34.837458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:34.837824 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:34.837981 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:34.838715 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 04:14:34.838917 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":215,"threads_started":1}
I20260807 04:14:34.840447 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.821266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:34705 (local address 127.18.193.3:37581)
0807 04:14:34.821975 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:34.822023 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:34.824083 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:34.824918 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:34.824930 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:34.825674 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:34.826437 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.826469 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.828039 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.828048 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.829525 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.829538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.830358 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.830363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.830586 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.831218 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:34.831240 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:34.833415 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:34.836249 (+ 2834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:34.836258 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:34.836274 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:34.836771 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:34.837138 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:34.837143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:34.837146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:34.837324 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:34.837985 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:34.837992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:34.838308 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 04:14:34.838564 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 04:14:34.839060 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":277,"threads_started":1}
I20260807 04:14:34.841842 6550 heartbeater.cc:347] Connected to a master server at 127.18.193.62:34705
I20260807 04:14:34.842134 6550 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:34.842839 6550 heartbeater.cc:511] Master 127.18.193.62:34705 requested a full tablet report, sending...
I20260807 04:14:34.844251 6044 ts_manager.cc:194] Registered new tserver with Master: 02cfbe4875c4483ab131a37869c6f582 (127.18.193.3:33497)
I20260807 04:14:34.845173 6044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:37581
I20260807 04:14:34.853720 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:14:34.871651 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.857887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.18.193.62:34705)
0807 04:14:34.858133 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:34.858143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:34.858519 (+ 376us) server_negotiation.cc:407] Connection header received
0807 04:14:34.858662 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:34.858665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:34.858732 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:34.858839 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:34.860243 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.861082 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.862503 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.862916 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.863023 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.865936 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:34.865941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:34.865944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:34.865990 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:34.868721 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:34.869495 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:34.869500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:34.869502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:34.869630 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:34.870058 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:34.870062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:34.870064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:34.870433 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:34.870643 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:34.871057 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 04:14:34.871209 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:14:34.876302 6044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:14:34.883248 6044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:14:34.914256 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.896517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:44733 (local address 127.0.0.1:36120)
0807 04:14:34.904786 (+ 8269us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:34.904859 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:34.905156 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:34.906068 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:34.906073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:34.906096 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:34.906733 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.906745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.908788 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.908793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.911160 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.911176 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.913168 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.913177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.913381 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.913434 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 04:14:34.913565 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 04:14:34.913751 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8073,"mutex_wait_us":30,"spinlock_wait_cycles":36224,"thread_start_us":7824,"threads_started":1}
I20260807 04:14:34.914254 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.896856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.18.193.2:44733)
0807 04:14:34.904168 (+ 7312us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:34.904180 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:34.904968 (+ 788us) server_negotiation.cc:407] Connection header received
0807 04:14:34.905434 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:34.905443 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:34.905689 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:34.905821 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:34.907002 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.908635 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.911339 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.912900 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.913037 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.913679 (+ 642us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:34.913747 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 04:14:34.913925 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7100,"thread_start_us":6967,"threads_started":1}
I20260807 04:14:34.914881 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.896219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.18.193.1:42083)
0807 04:14:34.903562 (+ 7343us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:34.903576 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:34.904979 (+ 1403us) server_negotiation.cc:407] Connection header received
0807 04:14:34.905411 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:34.905420 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:34.905694 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:34.905854 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:34.907070 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.908303 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.912083 (+ 3780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.913370 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.913507 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.913731 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:34.914377 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 04:14:34.914546 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7091,"thread_start_us":6899,"threads_started":1}
I20260807 04:14:34.915709 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.905830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.18.193.3:33497)
0807 04:14:34.906663 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:34.906673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:34.907356 (+ 683us) server_negotiation.cc:407] Connection header received
0807 04:14:34.907680 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:34.907688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:34.907880 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:34.908041 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:34.909055 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.910455 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.913037 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.914515 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.914633 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.914839 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:34.915206 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 04:14:34.915373 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":463,"threads_started":1}
I20260807 04:14:34.915830 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.895825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42083 (local address 127.0.0.1:33030)
0807 04:14:34.904786 (+ 8961us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:34.904861 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:34.905157 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:34.906218 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:34.906230 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:34.906267 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:34.906814 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.906832 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.908472 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.908481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.911897 (+ 3416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.911915 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.913958 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.913965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.914158 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.914190 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:14:34.915243 (+ 1053us) client_negotiation.cc:240] Negotiation successful
0807 04:14:34.915391 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8682,"mutex_wait_us":913,"spinlock_wait_cycles":20096,"thread_start_us":8435,"threads_started":1}
I20260807 04:14:34.916356 6569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:34.906656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33497 (local address 127.0.0.1:51476)
0807 04:14:34.907251 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:34.907269 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:34.907455 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:34.908312 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:34.908316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:34.908338 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:34.908843 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.908857 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.910625 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.910630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.912843 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:34.912860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:34.914824 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:34.914831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:34.914999 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:34.915036 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 04:14:34.915926 (+ 890us) client_negotiation.cc:240] Negotiation successful
0807 04:14:34.916085 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":30,"thread_start_us":303,"threads_started":1}
I20260807 04:14:34.919415 6323 tablet_service.cc:1467] Processing CreateTablet for tablet d91ee5e135864f3dbfe6554628b541c9 (DEFAULT_TABLE table=test-table [id=a58a7da7573b44bd85e76060632f15e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:34.919415 6179 tablet_service.cc:1467] Processing CreateTablet for tablet d91ee5e135864f3dbfe6554628b541c9 (DEFAULT_TABLE table=test-table [id=a58a7da7573b44bd85e76060632f15e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:34.919955 6476 tablet_service.cc:1467] Processing CreateTablet for tablet d91ee5e135864f3dbfe6554628b541c9 (DEFAULT_TABLE table=test-table [id=a58a7da7573b44bd85e76060632f15e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:34.920961 6323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d91ee5e135864f3dbfe6554628b541c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:34.920960 6179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d91ee5e135864f3dbfe6554628b541c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:34.921247 6476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d91ee5e135864f3dbfe6554628b541c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:34.933357 6570 tablet_bootstrap.cc:492] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Bootstrap starting.
I20260807 04:14:34.935741 6572 tablet_bootstrap.cc:492] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Bootstrap starting.
I20260807 04:14:34.938124 6570 tablet_bootstrap.cc:654] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:34.939390 6572 tablet_bootstrap.cc:654] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:34.939677 6570 log.cc:824] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:34.940764 6572 log.cc:824] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:34.943816 6570 tablet_bootstrap.cc:492] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: No bootstrap required, opened a new log
I20260807 04:14:34.941392 6573 tablet_bootstrap.cc:492] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Bootstrap starting.
I20260807 04:14:34.944176 6570 ts_tablet_manager.cc:1397] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260807 04:14:34.947009 6572 tablet_bootstrap.cc:492] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: No bootstrap required, opened a new log
I20260807 04:14:34.947299 6572 ts_tablet_manager.cc:1397] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260807 04:14:34.948221 6573 tablet_bootstrap.cc:654] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:34.949326 6573 log.cc:824] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:34.951930 6573 tablet_bootstrap.cc:492] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: No bootstrap required, opened a new log
I20260807 04:14:34.952210 6573 ts_tablet_manager.cc:1397] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260807 04:14:34.955325 6570 raft_consensus.cc:348] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:34.955801 6570 raft_consensus.cc:374] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:34.955914 6570 raft_consensus.cc:729] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02cfbe4875c4483ab131a37869c6f582, State: Initialized, Role: FOLLOWER
I20260807 04:14:34.956636 6570 consensus_queue.cc:260] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:34.958340 6572 raft_consensus.cc:348] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:34.958936 6572 raft_consensus.cc:374] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:34.959070 6572 raft_consensus.cc:729] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12d29e9390144b085342c4cb672cdec, State: Initialized, Role: FOLLOWER
I20260807 04:14:34.959888 6572 consensus_queue.cc:260] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:34.962287 6572 ts_tablet_manager.cc:1428] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260807 04:14:34.962231 6573 raft_consensus.cc:348] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:34.962798 6573 raft_consensus.cc:374] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:34.962954 6573 raft_consensus.cc:729] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 364501ac3f8241efa9b0a3060b964861, State: Initialized, Role: FOLLOWER
I20260807 04:14:34.963330 6550 heartbeater.cc:503] Master 127.18.193.62:34705 was elected leader, sending a full tablet report...
I20260807 04:14:34.963629 6573 consensus_queue.cc:260] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:34.964733 6395 heartbeater.cc:503] Master 127.18.193.62:34705 was elected leader, sending a full tablet report...
I20260807 04:14:34.965615 6570 ts_tablet_manager.cc:1428] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Time spent starting tablet: real 0.021s user 0.018s sys 0.001s
I20260807 04:14:34.966527 6573 ts_tablet_manager.cc:1428] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
W20260807 04:14:34.985431 6250 tablet.cc:2369] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:14:35.017058 6396 tablet.cc:2369] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:14:35.069769 6551 tablet.cc:2369] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:35.082346 6581 raft_consensus.cc:482] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:35.082738 6581 raft_consensus.cc:504] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:35.084707 6581 leader_election.cc:288] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d12d29e9390144b085342c4cb672cdec (127.18.193.1:42083), 02cfbe4875c4483ab131a37869c6f582 (127.18.193.3:33497)
I20260807 04:14:35.093425 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.085778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:41779 (local address 127.18.193.3:33497)
0807 04:14:35.086375 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:35.086384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:35.087061 (+ 677us) server_negotiation.cc:407] Connection header received
0807 04:14:35.087304 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:35.087309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:35.087379 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:35.087436 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:35.088253 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.089478 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.091462 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.092397 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.092594 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.092674 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:35.093018 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 04:14:35.093155 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:14:35.093520 6583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.085269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:33497 (local address 127.18.193.2:41779)
0807 04:14:35.086943 (+ 1674us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:35.086970 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:35.087161 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:35.087629 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:35.087633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:35.087655 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:35.088037 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:35.088048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.089627 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.089636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:35.091283 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:35.091297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.092641 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.092648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:35.092823 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.092858 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:14:35.092971 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 04:14:35.093103 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1532,"mutex_wait_us":54,"thread_start_us":1314,"threads_started":1}
I20260807 04:14:35.094532 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.085208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:50767 (local address 127.18.193.1:42083)
0807 04:14:35.085439 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:35.085449 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:35.086768 (+ 1319us) server_negotiation.cc:407] Connection header received
0807 04:14:35.087086 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:35.087090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:35.087152 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:35.087259 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:35.088348 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.089473 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.092123 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.093338 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.093513 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.093602 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:35.094156 (+ 554us) server_negotiation.cc:299] Negotiation successful
0807 04:14:35.094296 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:14:35.094642 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.085173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42083 (local address 127.18.193.2:50767)
0807 04:14:35.086641 (+ 1468us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:35.086668 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:35.086904 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:35.087280 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:35.087285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:35.087311 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:35.087639 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:35.087648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.089679 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.089689 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:35.091936 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:35.091950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.093755 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.093763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:35.093978 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.094010 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:14:35.094110 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 04:14:35.094249 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"mutex_wait_us":25,"thread_start_us":1022,"threads_started":1}
I20260807 04:14:35.094753 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91ee5e135864f3dbfe6554628b541c9" candidate_uuid: "364501ac3f8241efa9b0a3060b964861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02cfbe4875c4483ab131a37869c6f582" is_pre_election: true
I20260807 04:14:35.095357 6498 raft_consensus.cc:2454] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 364501ac3f8241efa9b0a3060b964861 in term 0.
I20260807 04:14:35.095788 6199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91ee5e135864f3dbfe6554628b541c9" candidate_uuid: "364501ac3f8241efa9b0a3060b964861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12d29e9390144b085342c4cb672cdec" is_pre_election: true
I20260807 04:14:35.096153 6276 leader_election.cc:302] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02cfbe4875c4483ab131a37869c6f582, 364501ac3f8241efa9b0a3060b964861; no voters:
I20260807 04:14:35.096382 6199 raft_consensus.cc:2454] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 364501ac3f8241efa9b0a3060b964861 in term 0.
I20260807 04:14:35.096648 6581 raft_consensus.cc:2781] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:14:35.096814 6581 raft_consensus.cc:482] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:14:35.096949 6581 raft_consensus.cc:3037] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:35.099225 6581 raft_consensus.cc:504] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:35.100015 6581 leader_election.cc:288] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [CANDIDATE]: Term 1 election: Requested vote from peers d12d29e9390144b085342c4cb672cdec (127.18.193.1:42083), 02cfbe4875c4483ab131a37869c6f582 (127.18.193.3:33497)
I20260807 04:14:35.100392 6199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91ee5e135864f3dbfe6554628b541c9" candidate_uuid: "364501ac3f8241efa9b0a3060b964861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12d29e9390144b085342c4cb672cdec"
I20260807 04:14:35.100463 6498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91ee5e135864f3dbfe6554628b541c9" candidate_uuid: "364501ac3f8241efa9b0a3060b964861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02cfbe4875c4483ab131a37869c6f582"
I20260807 04:14:35.100623 6199 raft_consensus.cc:3037] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:35.100746 6498 raft_consensus.cc:3037] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:35.102908 6199 raft_consensus.cc:2454] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 364501ac3f8241efa9b0a3060b964861 in term 1.
I20260807 04:14:35.102954 6498 raft_consensus.cc:2454] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 364501ac3f8241efa9b0a3060b964861 in term 1.
I20260807 04:14:35.103329 6274 leader_election.cc:302] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 364501ac3f8241efa9b0a3060b964861, d12d29e9390144b085342c4cb672cdec; no voters:
I20260807 04:14:35.103662 6581 raft_consensus.cc:2781] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:35.104115 6581 raft_consensus.cc:686] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 1 LEADER]: Becoming Leader. State: Replica: 364501ac3f8241efa9b0a3060b964861, State: Running, Role: LEADER
I20260807 04:14:35.104686 6581 consensus_queue.cc:237] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } }
I20260807 04:14:35.109611 6044 catalog_manager.cc:5495] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 reported cstate change: term changed from 0 to 1, leader changed from <none> to 364501ac3f8241efa9b0a3060b964861 (127.18.193.2). New cstate: current_term: 1 leader_uuid: "364501ac3f8241efa9b0a3060b964861" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "364501ac3f8241efa9b0a3060b964861" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 44733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 } health_report { overall_health: UNKNOWN } } }
I20260807 04:14:35.158923 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.153225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.18.193.2:44733)
0807 04:14:35.153426 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:35.153431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:35.153508 (+ 77us) server_negotiation.cc:407] Connection header received
0807 04:14:35.153706 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:35.153710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:35.153789 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:35.153925 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:14:35.154366 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.155190 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.156688 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.157107 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.157228 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.157730 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:14:35.158253 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:14:35.158388 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:35.158531 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 04:14:35.158605 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:14:35.167450 6199 raft_consensus.cc:1261] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 1 FOLLOWER]: Refusing update from remote peer 364501ac3f8241efa9b0a3060b964861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:35.167435 6498 raft_consensus.cc:1261] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 1 FOLLOWER]: Refusing update from remote peer 364501ac3f8241efa9b0a3060b964861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:35.168453 6584 consensus_queue.cc:1035] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02cfbe4875c4483ab131a37869c6f582" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 33497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:35.168851 6581 consensus_queue.cc:1035] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d12d29e9390144b085342c4cb672cdec" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:35.181269 6589 mvcc.cc:205] Tried to move back new op lower bound from 7315767603875471360 to 7315767603632549888. Current Snapshot: MvccSnapshot[applied={T|T < 7315767603875471360}]
I20260807 04:14:35.182717 6590 mvcc.cc:205] Tried to move back new op lower bound from 7315767603875471360 to 7315767603632549888. Current Snapshot: MvccSnapshot[applied={T|T < 7315767603875471360}]
I20260807 04:14:35.194526 6043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:35.194855 6043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:35.198776 6043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9737add050e6421988ef011d1d2d5b6f: Sending DeleteTablet for 3 replicas of tablet d91ee5e135864f3dbfe6554628b541c9
I20260807 04:14:35.200199 6179 tablet_service.cc:1514] Processing DeleteTablet for tablet d91ee5e135864f3dbfe6554628b541c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:35 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:33030
I20260807 04:14:35.200206 6323 tablet_service.cc:1514] Processing DeleteTablet for tablet d91ee5e135864f3dbfe6554628b541c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:35 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:36120
I20260807 04:14:35.200666 6476 tablet_service.cc:1514] Processing DeleteTablet for tablet d91ee5e135864f3dbfe6554628b541c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:35 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:51476
I20260807 04:14:35.205817 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 6112
I20260807 04:14:35.215425 6600 tablet_replica.cc:330] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: stopping tablet replica
I20260807 04:14:35.215902 6600 raft_consensus.cc:2229] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:35.216465 6600 raft_consensus.cc:2258] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:35.218041 6603 tablet_replica.cc:330] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: stopping tablet replica
I20260807 04:14:35.218447 6603 raft_consensus.cc:2229] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:35.218910 6603 raft_consensus.cc:2258] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:35.221333 6603 ts_tablet_manager.cc:1905] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:35.221351 6600 ts_tablet_manager.cc:1905] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:35.222549 6601 tablet_replica.cc:330] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: stopping tablet replica
I20260807 04:14:35.222990 6601 raft_consensus.cc:2229] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:14:35.224030 6601 raft_consensus.cc:2258] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 04:14:35 dist-test-slave-8n49 krb5kdc[5998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:14:35.229300 6603 ts_tablet_manager.cc:1918] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:35.229573 6603 log.cc:1196] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/d91ee5e135864f3dbfe6554628b541c9
I20260807 04:14:35.230212 6603 ts_tablet_manager.cc:1939] T d91ee5e135864f3dbfe6554628b541c9 P 02cfbe4875c4483ab131a37869c6f582: Deleting consensus metadata
I20260807 04:14:35.230723 6601 ts_tablet_manager.cc:1905] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:35.232465 6030 catalog_manager.cc:4841] TS 02cfbe4875c4483ab131a37869c6f582 (127.18.193.3:33497): tablet d91ee5e135864f3dbfe6554628b541c9 (table test-table [id=a58a7da7573b44bd85e76060632f15e9]) successfully deleted
I20260807 04:14:35.234990 6600 ts_tablet_manager.cc:1918] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:35.235219 6600 log.cc:1196] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/d91ee5e135864f3dbfe6554628b541c9
I20260807 04:14:35.235926 6600 ts_tablet_manager.cc:1939] T d91ee5e135864f3dbfe6554628b541c9 P d12d29e9390144b085342c4cb672cdec: Deleting consensus metadata
I20260807 04:14:35.237975 6029 catalog_manager.cc:4841] TS d12d29e9390144b085342c4cb672cdec (127.18.193.1:42083): tablet d91ee5e135864f3dbfe6554628b541c9 (table test-table [id=a58a7da7573b44bd85e76060632f15e9]) successfully deleted
I20260807 04:14:35.238296 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.207099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.18.193.1:42083)
0807 04:14:35.207284 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:35.207291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:35.217388 (+ 10097us) server_negotiation.cc:407] Connection header received
0807 04:14:35.217619 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:35.217629 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:35.217743 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:35.217862 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:35.220324 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.221464 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.223402 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.224064 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.224174 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.230298 (+ 6124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:35.230313 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:35.230332 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:35.230383 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:35.234418 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:35.235324 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:35.235332 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:35.235338 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:35.235582 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:35.236107 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:35.236113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:35.236116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:35.236613 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:35.236907 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:35.237540 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 04:14:35.237915 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:14:35.239107 6601 ts_tablet_manager.cc:1918] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:35.239291 6601 log.cc:1196] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/d91ee5e135864f3dbfe6554628b541c9
I20260807 04:14:35.239698 6243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:35.243690 6601 ts_tablet_manager.cc:1939] T d91ee5e135864f3dbfe6554628b541c9 P 364501ac3f8241efa9b0a3060b964861: Deleting consensus metadata
I20260807 04:14:35.245500 6030 catalog_manager.cc:4841] TS 364501ac3f8241efa9b0a3060b964861 (127.18.193.2:44733): tablet d91ee5e135864f3dbfe6554628b541c9 (table test-table [id=a58a7da7573b44bd85e76060632f15e9]) successfully deleted
I20260807 04:14:35.348631 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 6112
I20260807 04:14:35.368265 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 6255
Aug 07 04:14:35 dist-test-slave-8n49 krb5kdc[5998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:14:35.387180 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.369181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36150 (local address 127.18.193.2:44733)
0807 04:14:35.369709 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:35.369714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:35.369729 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:14:35.369788 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:35.369792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:35.369848 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:35.369991 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:35.372004 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.373066 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.374756 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.375459 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.375603 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.380795 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:35.380809 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:35.380824 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:35.380891 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:35.384253 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:35.385161 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:35.385169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:35.385171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:35.385321 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:35.385704 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:35.385707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:35.385709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:35.386092 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:35.386352 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:35.386649 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 04:14:35.386818 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415}
I20260807 04:14:35.388037 6390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:35.505307 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 6255
I20260807 04:14:35.527526 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 6402
Aug 07 04:14:35 dist-test-slave-8n49 krb5kdc[5998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:14:35.543700 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:35.528421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.18.193.3:33497)
0807 04:14:35.528661 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:35.528665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:35.528684 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:14:35.528852 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:35.528856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:35.528933 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:35.529032 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:35.530576 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.531374 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.532713 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:35.533161 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:35.533260 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:35.538037 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:35.538052 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:35.538069 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:35.538106 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:35.541112 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:35.541818 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:35.541825 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:35.541828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:35.541960 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:35.542287 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:35.542290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:35.542292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:35.542637 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:35.542848 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:35.543141 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 04:14:35.543302 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:14:35.544401 6545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:35.635248 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 6402
I20260807 04:14:35.654433 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 6012
I20260807 04:14:35.655660 6078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:35.767149 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 6012
2026-08-07T04:14:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4609 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T04:14:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:35Z Disabled control of system clock
I20260807 04:14:35.856752 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35735
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:35735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:14:36.198952 6612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:36.199232 6612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:36.199321 6612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:36.206643 6612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:36.206758 6612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:36.206797 6612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:36.206856 6612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:36.206904 6612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:36.216248 6612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:35735
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:36.217577 6612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:36.219331 6612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:36.231238 6619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:36.231405 6618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:36.231335 6622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:36.232474 6612 server_base.cc:1034] running on GCE node
I20260807 04:14:36.233281 6612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:36.234736 6612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:36.235934 6612 hybrid_clock.cc:648] HybridClock initialized: now 1786076076235872 us; error 60 us; skew 500 ppm
I20260807 04:14:36.236300 6612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:36.238675 6612 webserver.cc:460] Webserver started at http://127.18.193.62:39455/ using document root <none> and password file <none>
I20260807 04:14:36.239372 6612 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:36.239466 6612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:36.239799 6612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:36.242882 6612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "60851e5e726848e3ac8013fad54ee341"
format_stamp: "Formatted at 2026-08-07 04:14:36 on dist-test-slave-8n49"
server_key: "5283daeb9178240c6a8e943c2483baa3"
server_key_iv: "8e948063c55b75c0568b359f1b6de56e"
server_key_version: "encryptionkey@0"
I20260807 04:14:36.243839 6612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "60851e5e726848e3ac8013fad54ee341"
format_stamp: "Formatted at 2026-08-07 04:14:36 on dist-test-slave-8n49"
server_key: "5283daeb9178240c6a8e943c2483baa3"
server_key_iv: "8e948063c55b75c0568b359f1b6de56e"
server_key_version: "encryptionkey@0"
I20260807 04:14:36.249188 6612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:14:36.253391 6629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:36.254882 6612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260807 04:14:36.255125 6612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "60851e5e726848e3ac8013fad54ee341"
format_stamp: "Formatted at 2026-08-07 04:14:36 on dist-test-slave-8n49"
server_key: "5283daeb9178240c6a8e943c2483baa3"
server_key_iv: "8e948063c55b75c0568b359f1b6de56e"
server_key_version: "encryptionkey@0"
I20260807 04:14:36.255443 6612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:36.297237 6612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:36.298179 6612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:36.298512 6612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:36.318670 6612 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:35735
I20260807 04:14:36.318712 6684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:35735 every 8 connection(s)
I20260807 04:14:36.320098 6612 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:36.324059 6685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:36.325374 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 6612
I20260807 04:14:36.325579 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:36.326061 19204 external_mini_cluster.cc:1428] Setting key 78a9f0c1bb520e2640a4be160ea99089
I20260807 04:14:36.339536 6685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341: Bootstrap starting.
I20260807 04:14:36.340660 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:36.328046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.18.193.62:35735)
0807 04:14:36.329156 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:36.329168 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:36.329214 (+ 46us) server_negotiation.cc:407] Connection header received
0807 04:14:36.330826 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:36.330848 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:36.331307 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:36.331632 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:36.332338 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:36.333694 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:36.335844 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:36.336622 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:36.336777 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:36.338223 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:36.338231 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:36.338245 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:36.338785 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:36.338878 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:36.338899 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:36.338998 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:36.339275 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 04:14:36.339618 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":477,"threads_started":1}
I20260807 04:14:36.343531 6685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:36.344682 6685 log.cc:824] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:36.347684 6685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341: No bootstrap required, opened a new log
I20260807 04:14:36.355746 6685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60851e5e726848e3ac8013fad54ee341" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35735 } }
I20260807 04:14:36.356127 6685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:36.356207 6685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60851e5e726848e3ac8013fad54ee341, State: Initialized, Role: FOLLOWER
I20260807 04:14:36.356858 6685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60851e5e726848e3ac8013fad54ee341" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35735 } }
I20260807 04:14:36.357100 6685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:36.357208 6685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:36.357340 6685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:36.360136 6685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60851e5e726848e3ac8013fad54ee341" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35735 } }
I20260807 04:14:36.360700 6685 leader_election.cc:302] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60851e5e726848e3ac8013fad54ee341; no voters:
I20260807 04:14:36.361265 6685 leader_election.cc:288] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:36.361403 6690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:36.362177 6690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [term 1 LEADER]: Becoming Leader. State: Replica: 60851e5e726848e3ac8013fad54ee341, State: Running, Role: LEADER
I20260807 04:14:36.362740 6690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60851e5e726848e3ac8013fad54ee341" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35735 } }
I20260807 04:14:36.363379 6685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:36.365839 6691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60851e5e726848e3ac8013fad54ee341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60851e5e726848e3ac8013fad54ee341" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35735 } } }
I20260807 04:14:36.366369 6691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:36.367817 6693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60851e5e726848e3ac8013fad54ee341. Latest consensus state: current_term: 1 leader_uuid: "60851e5e726848e3ac8013fad54ee341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60851e5e726848e3ac8013fad54ee341" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35735 } } }
I20260807 04:14:36.368269 6693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:36.369475 6700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:36.373718 6700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:36.382706 6700 catalog_manager.cc:1347] Generated new cluster ID: ac1616b282634753943afaf207bf9aec
I20260807 04:14:36.382825 6700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:36.395591 6700 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:36.396936 6700 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:36.410460 6700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341: Generated new TSK 0
I20260807 04:14:36.411361 6700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:36.424439 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35735
--builtin_ntp_servers=127.18.193.20:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:14:36.778885 6710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:36.779194 6710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:36.779379 6710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:36.787283 6710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:36.787390 6710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:36.787565 6710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:14:36.796844 6710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:36.798189 6710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:36.799988 6710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:36.813989 6715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:36.814078 6716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:36.814028 6718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:36.820137 6710 server_base.cc:1034] running on GCE node
I20260807 04:14:36.820960 6710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:36.822000 6710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:36.823251 6710 hybrid_clock.cc:648] HybridClock initialized: now 1786076076823192 us; error 53 us; skew 500 ppm
I20260807 04:14:36.823616 6710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:36.825977 6710 webserver.cc:460] Webserver started at http://127.18.193.1:44347/ using document root <none> and password file <none>
I20260807 04:14:36.826668 6710 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:36.826805 6710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:36.827128 6710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:36.830075 6710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "d323c21e9535447695ff8981d6822dee"
format_stamp: "Formatted at 2026-08-07 04:14:36 on dist-test-slave-8n49"
server_key: "7d22394a0efc88a9e3d051dbbea869d6"
server_key_iv: "27d4f6475807d38e718289105e6904da"
server_key_version: "encryptionkey@0"
I20260807 04:14:36.830878 6710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "d323c21e9535447695ff8981d6822dee"
format_stamp: "Formatted at 2026-08-07 04:14:36 on dist-test-slave-8n49"
server_key: "7d22394a0efc88a9e3d051dbbea869d6"
server_key_iv: "27d4f6475807d38e718289105e6904da"
server_key_version: "encryptionkey@0"
I20260807 04:14:36.836059 6710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:14:36.839622 6724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:36.841302 6710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:36.841509 6710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "d323c21e9535447695ff8981d6822dee"
format_stamp: "Formatted at 2026-08-07 04:14:36 on dist-test-slave-8n49"
server_key: "7d22394a0efc88a9e3d051dbbea869d6"
server_key_iv: "27d4f6475807d38e718289105e6904da"
server_key_version: "encryptionkey@0"
I20260807 04:14:36.841737 6710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:36.873248 6710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:36.874186 6710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:36.874586 6710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:36.875635 6710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:36.877202 6710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:36.877290 6710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:36.877374 6710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:36.877429 6710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:36.911762 6710 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:35155
I20260807 04:14:36.911871 6848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:35155 every 8 connection(s)
I20260807 04:14:36.913177 6710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:14:36.915298 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 6710
I20260807 04:14:36.915483 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:14:36.915993 19204 external_mini_cluster.cc:1428] Setting key 5708136024d6a283c9fa7bf1948243fc
I20260807 04:14:36.919348 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35735
--builtin_ntp_servers=127.18.193.20:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:14:36.927459 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:36.916860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:59197 (local address 127.18.193.62:35735)
0807 04:14:36.917318 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:36.917324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:36.917658 (+ 334us) server_negotiation.cc:407] Connection header received
0807 04:14:36.919748 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:36.919756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:36.919855 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:36.919987 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:36.921208 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:36.922347 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:36.924118 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:36.924852 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:36.925009 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:36.926438 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:36.926446 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:36.926455 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:36.926482 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:36.926537 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:36.926556 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:36.926651 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:36.926884 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 04:14:36.927008 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":228,"threads_started":1}
I20260807 04:14:36.928213 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:36.916342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35735 (local address 127.18.193.1:59197)
0807 04:14:36.917479 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:36.917538 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:36.919429 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:36.920232 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:36.920248 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:36.920338 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:36.921032 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:36.921051 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:36.922554 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:36.922559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:36.923956 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:36.923970 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:36.925081 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:36.925086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:36.925304 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:36.926104 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:36.926108 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:36.926147 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:36.926232 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:36.926660 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:36.926668 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:36.926673 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:14:36.926821 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 04:14:36.927117 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":359,"threads_started":1}
I20260807 04:14:36.929908 6849 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35735
I20260807 04:14:36.930259 6849 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:36.931173 6849 heartbeater.cc:511] Master 127.18.193.62:35735 requested a full tablet report, sending...
I20260807 04:14:36.933533 6648 ts_manager.cc:194] Registered new tserver with Master: d323c21e9535447695ff8981d6822dee (127.18.193.1:35155)
I20260807 04:14:36.935140 6648 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.1:59197
W20260807 04:14:37.269625 6853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:37.269901 6853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:37.270041 6853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:37.277400 6853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:37.277498 6853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:37.277701 6853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:14:37.286871 6853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:37.288163 6853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:37.289933 6853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:37.303174 6861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:37.303249 6858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:37.303217 6859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:37.303731 6853 server_base.cc:1034] running on GCE node
I20260807 04:14:37.304647 6853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:37.305583 6853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:37.306787 6853 hybrid_clock.cc:648] HybridClock initialized: now 1786076077306752 us; error 30 us; skew 500 ppm
I20260807 04:14:37.307063 6853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:37.309394 6853 webserver.cc:460] Webserver started at http://127.18.193.2:33393/ using document root <none> and password file <none>
I20260807 04:14:37.310050 6853 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:37.310139 6853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:37.310388 6853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:37.313398 6853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "f074f016930e466884813151992106e9"
format_stamp: "Formatted at 2026-08-07 04:14:37 on dist-test-slave-8n49"
server_key: "b6730a835fe6d592a5cb0b5e9506af68"
server_key_iv: "f20014813c7e0abdad66a4d4b5267ce2"
server_key_version: "encryptionkey@0"
I20260807 04:14:37.314131 6853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "f074f016930e466884813151992106e9"
format_stamp: "Formatted at 2026-08-07 04:14:37 on dist-test-slave-8n49"
server_key: "b6730a835fe6d592a5cb0b5e9506af68"
server_key_iv: "f20014813c7e0abdad66a4d4b5267ce2"
server_key_version: "encryptionkey@0"
I20260807 04:14:37.319118 6853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:14:37.322669 6867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:37.324010 6853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 04:14:37.324183 6853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "f074f016930e466884813151992106e9"
format_stamp: "Formatted at 2026-08-07 04:14:37 on dist-test-slave-8n49"
server_key: "b6730a835fe6d592a5cb0b5e9506af68"
server_key_iv: "f20014813c7e0abdad66a4d4b5267ce2"
server_key_version: "encryptionkey@0"
I20260807 04:14:37.324391 6853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:37.355130 6853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:37.356014 6853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:37.356326 6853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:37.357728 6853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:37.359691 6853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:37.359870 6853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:37.360074 6853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:37.360227 6853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:37.400451 6853 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:46863
I20260807 04:14:37.400583 6995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:46863 every 8 connection(s)
I20260807 04:14:37.401798 6853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:14:37.404721 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 6853
I20260807 04:14:37.404928 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:14:37.405382 19204 external_mini_cluster.cc:1428] Setting key 9c5920a975ccffb88fe12174bf2c8542
I20260807 04:14:37.408169 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35735
--builtin_ntp_servers=127.18.193.20:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:14:37.414592 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.404414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:47787 (local address 127.18.193.62:35735)
0807 04:14:37.405544 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:37.405551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:37.406233 (+ 682us) server_negotiation.cc:407] Connection header received
0807 04:14:37.408068 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:37.408073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:37.408190 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:37.408316 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:37.409506 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.410245 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.411777 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.412182 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.412320 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.413771 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:37.413779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:37.413788 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:37.413807 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:37.413839 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:37.413850 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:37.413957 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:37.414175 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 04:14:37.414276 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":958}
I20260807 04:14:37.415663 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.405038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35735 (local address 127.18.193.2:47787)
0807 04:14:37.406079 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:37.406129 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:37.407809 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:37.408612 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:37.408625 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:37.408753 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:37.409321 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.409338 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.410400 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.410413 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.411632 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.411644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.412429 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.412437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.412727 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.413440 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:37.413450 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:37.413489 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:37.413565 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:37.413949 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:37.413962 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:37.413972 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:14:37.414146 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 04:14:37.414544 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":436,"threads_started":1}
I20260807 04:14:37.417135 6997 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35735
I20260807 04:14:37.417485 6997 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:37.418390 6997 heartbeater.cc:511] Master 127.18.193.62:35735 requested a full tablet report, sending...
I20260807 04:14:37.419986 6648 ts_manager.cc:194] Registered new tserver with Master: f074f016930e466884813151992106e9 (127.18.193.2:46863)
I20260807 04:14:37.420926 6648 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.2:47787
W20260807 04:14:37.747041 7001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:37.747370 7001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:37.747540 7001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:37.754841 7001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:37.754987 7001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:37.755162 7001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:14:37.765105 7001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:37.766446 7001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:37.768287 7001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:37.781316 7006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:37.781286 7009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:37.781270 7007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:37.782469 7001 server_base.cc:1034] running on GCE node
I20260807 04:14:37.783108 7001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:37.784081 7001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:37.785282 7001 hybrid_clock.cc:648] HybridClock initialized: now 1786076077785243 us; error 62 us; skew 500 ppm
I20260807 04:14:37.785599 7001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:37.787922 7001 webserver.cc:460] Webserver started at http://127.18.193.3:42481/ using document root <none> and password file <none>
I20260807 04:14:37.788705 7001 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:37.788825 7001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:37.789153 7001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:37.791960 7001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "095a234565e84f9881c132fe2d1703d0"
format_stamp: "Formatted at 2026-08-07 04:14:37 on dist-test-slave-8n49"
server_key: "8cc7d2ab70aa9727ec86e974b4e0c367"
server_key_iv: "695ee4becde1c578b43169e52b09a8d6"
server_key_version: "encryptionkey@0"
I20260807 04:14:37.792838 7001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "095a234565e84f9881c132fe2d1703d0"
format_stamp: "Formatted at 2026-08-07 04:14:37 on dist-test-slave-8n49"
server_key: "8cc7d2ab70aa9727ec86e974b4e0c367"
server_key_iv: "695ee4becde1c578b43169e52b09a8d6"
server_key_version: "encryptionkey@0"
I20260807 04:14:37.797901 7001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:14:37.801318 7017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:37.802639 7001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:14:37.802816 7001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "095a234565e84f9881c132fe2d1703d0"
format_stamp: "Formatted at 2026-08-07 04:14:37 on dist-test-slave-8n49"
server_key: "8cc7d2ab70aa9727ec86e974b4e0c367"
server_key_iv: "695ee4becde1c578b43169e52b09a8d6"
server_key_version: "encryptionkey@0"
I20260807 04:14:37.803038 7001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:37.815405 7001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:37.816197 7001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:37.816501 7001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:37.817559 7001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:37.819002 7001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:37.819094 7001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:37.819255 7001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:37.819320 7001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:37.854417 7001 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:45895
I20260807 04:14:37.854513 7142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:45895 every 8 connection(s)
I20260807 04:14:37.855763 7001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:14:37.859179 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 7001
I20260807 04:14:37.859367 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:14:37.860034 19204 external_mini_cluster.cc:1428] Setting key a6edf8815a80bd0dc6acc35e9ecae94d
I20260807 04:14:37.869045 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.858609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:51455 (local address 127.18.193.62:35735)
0807 04:14:37.858819 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:37.858824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:37.860413 (+ 1589us) server_negotiation.cc:407] Connection header received
0807 04:14:37.862616 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:37.862621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:37.862704 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:37.862821 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:37.863871 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.864731 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.866220 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.866649 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.866841 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.868141 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:37.868147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:37.868158 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:37.868175 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:37.868205 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:37.868217 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:37.868310 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:37.868589 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 04:14:37.868687 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:14:37.869944 7146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.859204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35735 (local address 127.18.193.3:51455)
0807 04:14:37.860258 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:37.860293 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:37.862350 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:37.863043 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:37.863051 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:37.863138 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:37.863700 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.863715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.864878 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.864882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.866076 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.866088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.866923 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.866933 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.867165 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.867823 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:37.867826 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:37.867873 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:37.867949 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:37.868309 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:37.868321 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:37.868331 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:14:37.868558 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 04:14:37.868944 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":325,"threads_started":1}
I20260807 04:14:37.871302 7143 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35735
I20260807 04:14:37.871658 7143 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:37.872816 7143 heartbeater.cc:511] Master 127.18.193.62:35735 requested a full tablet report, sending...
I20260807 04:14:37.874259 6648 ts_manager.cc:194] Registered new tserver with Master: 095a234565e84f9881c132fe2d1703d0 (127.18.193.3:45895)
I20260807 04:14:37.875348 6648 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.3:51455
I20260807 04:14:37.875402 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:14:37.885584 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.879502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.18.193.62:35735)
0807 04:14:37.879680 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:37.879685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:37.880206 (+ 521us) server_negotiation.cc:407] Connection header received
0807 04:14:37.880321 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:37.880325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:37.880420 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:37.880512 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:37.881064 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.881754 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.883290 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.883749 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.883843 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.884995 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:37.884999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:37.885002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:37.885019 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:37.885048 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:37.885059 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:37.885166 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:37.885322 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 04:14:37.885365 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:14:37.889480 6648 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:14:37.896292 6648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:14:37.922839 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.909095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.18.193.1:35155)
0807 04:14:37.913138 (+ 4043us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:37.913147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:37.913182 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:14:37.913302 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:37.913312 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:37.913554 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:37.914490 (+ 936us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:37.914766 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.916671 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.919782 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.921468 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.921659 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.921989 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:37.922243 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 04:14:37.922420 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3841,"thread_start_us":3686,"threads_started":1}
I20260807 04:14:37.922865 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.908872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:35155 (local address 127.0.0.1:46458)
0807 04:14:37.909755 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:37.909804 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:37.910001 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:37.913781 (+ 3780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:37.913786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:37.913818 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:37.914263 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.914280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.916856 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.916873 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.919527 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.919544 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.921733 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.921740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.921956 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.922018 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 04:14:37.922197 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 04:14:37.922423 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":81,"thread_start_us":1177,"threads_started":1}
I20260807 04:14:37.925280 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.909611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57742 (local address 127.18.193.3:45895)
0807 04:14:37.911901 (+ 2290us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:37.911910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:37.912429 (+ 519us) server_negotiation.cc:407] Connection header received
0807 04:14:37.912755 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:37.912767 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:37.913066 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:37.913285 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:37.915711 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.917578 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.921821 (+ 4243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.923315 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.923420 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.923675 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:37.924687 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0807 04:14:37.924873 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2096,"thread_start_us":2202,"threads_started":1}
I20260807 04:14:37.926081 7158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.909228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45895 (local address 127.0.0.1:57742)
0807 04:14:37.912280 (+ 3052us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:37.912299 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:37.912532 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:37.913619 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:37.913628 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:37.913661 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:37.915434 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.915450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.919132 (+ 3682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.919140 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.921588 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.921602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.924098 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.924107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.924355 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.924407 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 04:14:37.925529 (+ 1122us) client_negotiation.cc:240] Negotiation successful
0807 04:14:37.925691 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2921,"mutex_wait_us":4,"thread_start_us":3755,"threads_started":1}
I20260807 04:14:37.927588 6779 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe (DEFAULT_TABLE table=test-table [id=73395f93967446fdb878b73b051231d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:37.928599 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.913364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:46863 (local address 127.0.0.1:51028)
0807 04:14:37.921242 (+ 7878us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:37.921263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:37.921420 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:37.922339 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:37.922346 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:37.922372 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:37.922711 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.922723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.924420 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.924424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.925972 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:37.925985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.927830 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.927836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:37.928027 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.928051 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:14:37.928154 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 04:14:37.928290 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7732,"thread_start_us":7579,"threads_started":1}
I20260807 04:14:37.929637 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:37.913912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.18.193.2:46863)
0807 04:14:37.920765 (+ 6853us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:37.920775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:37.921381 (+ 606us) server_negotiation.cc:407] Connection header received
0807 04:14:37.921609 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:37.921618 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:37.921887 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:37.922095 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:37.922904 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.924243 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.926264 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:37.927579 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:37.927749 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:37.927974 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:37.928198 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 04:14:37.928364 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6671,"thread_start_us":6508,"threads_started":1}
I20260807 04:14:37.929658 6779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:37.930464 7068 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe (DEFAULT_TABLE table=test-table [id=73395f93967446fdb878b73b051231d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:37.932137 7068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:37.932492 6924 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe (DEFAULT_TABLE table=test-table [id=73395f93967446fdb878b73b051231d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:37.933640 6924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:37.938824 6849 heartbeater.cc:503] Master 127.18.193.62:35735 was elected leader, sending a full tablet report...
I20260807 04:14:37.945392 7162 tablet_bootstrap.cc:492] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Bootstrap starting.
I20260807 04:14:37.948462 7164 tablet_bootstrap.cc:492] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Bootstrap starting.
I20260807 04:14:37.949837 7163 tablet_bootstrap.cc:492] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Bootstrap starting.
I20260807 04:14:37.950196 7162 tablet_bootstrap.cc:654] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:37.951594 7162 log.cc:824] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:37.952739 7164 tablet_bootstrap.cc:654] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:37.953683 7163 tablet_bootstrap.cc:654] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:37.954455 7164 log.cc:824] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:37.954653 7163 log.cc:824] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:37.959440 7163 tablet_bootstrap.cc:492] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: No bootstrap required, opened a new log
I20260807 04:14:37.959448 7164 tablet_bootstrap.cc:492] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: No bootstrap required, opened a new log
I20260807 04:14:37.959450 7162 tablet_bootstrap.cc:492] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: No bootstrap required, opened a new log
I20260807 04:14:37.959724 7163 ts_tablet_manager.cc:1397] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 04:14:37.959735 7164 ts_tablet_manager.cc:1397] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260807 04:14:37.959735 7162 ts_tablet_manager.cc:1397] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260807 04:14:37.968001 7163 raft_consensus.cc:348] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:37.968429 7163 raft_consensus.cc:374] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:37.968626 7163 raft_consensus.cc:729] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d323c21e9535447695ff8981d6822dee, State: Initialized, Role: FOLLOWER
I20260807 04:14:37.969282 7163 consensus_queue.cc:260] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:37.970554 7162 raft_consensus.cc:348] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:37.970554 7164 raft_consensus.cc:348] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:37.970991 7164 raft_consensus.cc:374] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:37.971035 7162 raft_consensus.cc:374] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:37.971133 7163 ts_tablet_manager.cc:1428] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 04:14:37.971134 7162 raft_consensus.cc:729] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f074f016930e466884813151992106e9, State: Initialized, Role: FOLLOWER
I20260807 04:14:37.971148 7164 raft_consensus.cc:729] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095a234565e84f9881c132fe2d1703d0, State: Initialized, Role: FOLLOWER
I20260807 04:14:37.971777 7164 consensus_queue.cc:260] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:37.971740 7162 consensus_queue.cc:260] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:37.973382 7143 heartbeater.cc:503] Master 127.18.193.62:35735 was elected leader, sending a full tablet report...
I20260807 04:14:37.973631 7162 ts_tablet_manager.cc:1428] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260807 04:14:37.973729 7164 ts_tablet_manager.cc:1428] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260807 04:14:37.974185 6997 heartbeater.cc:503] Master 127.18.193.62:35735 was elected leader, sending a full tablet report...
I20260807 04:14:38.065896 7168 raft_consensus.cc:482] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:38.066323 7168 raft_consensus.cc:504] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:38.068416 7168 leader_election.cc:288] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095a234565e84f9881c132fe2d1703d0 (127.18.193.3:45895), f074f016930e466884813151992106e9 (127.18.193.2:46863)
I20260807 04:14:38.075373 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.068886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:56391 (local address 127.18.193.3:45895)
0807 04:14:38.069082 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:38.069087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:38.069662 (+ 575us) server_negotiation.cc:407] Connection header received
0807 04:14:38.069881 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:38.069890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:38.070018 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:38.070116 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:38.070732 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.071435 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.073301 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.074311 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.074607 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.074683 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:38.074978 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 04:14:38.075109 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:14:38.075506 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.068848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:45895 (local address 127.18.193.1:56391)
0807 04:14:38.069500 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:38.069522 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:38.069702 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:38.070179 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:38.070184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:38.070211 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:38.070571 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:38.070580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.071578 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.071585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:38.073107 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:38.073120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.074603 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.074609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:38.074792 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.074817 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:14:38.074951 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 04:14:38.075087 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":2,"thread_start_us":364,"threads_started":1}
I20260807 04:14:38.075577 7173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.069639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:46863 (local address 127.18.193.1:51453)
0807 04:14:38.070283 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:38.070297 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:38.070448 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:38.070811 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:38.070815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:38.070838 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:38.071181 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:38.071190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.072177 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.072183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:38.073598 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:38.073611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.074737 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.074744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:38.074894 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.074917 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:14:38.075025 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 04:14:38.075152 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":310,"threads_started":1}
I20260807 04:14:38.075450 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.069930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:51453 (local address 127.18.193.2:46863)
0807 04:14:38.070149 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:38.070155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:38.070394 (+ 239us) server_negotiation.cc:407] Connection header received
0807 04:14:38.070627 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:38.070636 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:38.070697 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:38.070797 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:38.071353 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.072032 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.073792 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.074538 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.074713 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.074783 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:38.075066 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 04:14:38.075195 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 04:14:38.076658 7092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6d7e9c7dba4d8ea9601227a3e51ebe" candidate_uuid: "d323c21e9535447695ff8981d6822dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095a234565e84f9881c132fe2d1703d0" is_pre_election: true
I20260807 04:14:38.076704 6946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6d7e9c7dba4d8ea9601227a3e51ebe" candidate_uuid: "d323c21e9535447695ff8981d6822dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f074f016930e466884813151992106e9" is_pre_election: true
I20260807 04:14:38.077198 6946 raft_consensus.cc:2454] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d323c21e9535447695ff8981d6822dee in term 0.
I20260807 04:14:38.077220 7092 raft_consensus.cc:2454] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d323c21e9535447695ff8981d6822dee in term 0.
I20260807 04:14:38.077916 6729 leader_election.cc:302] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d323c21e9535447695ff8981d6822dee, f074f016930e466884813151992106e9; no voters:
I20260807 04:14:38.078341 7168 raft_consensus.cc:2781] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:14:38.078523 7168 raft_consensus.cc:482] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:14:38.078668 7168 raft_consensus.cc:3037] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:38.081017 7168 raft_consensus.cc:504] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:38.081634 7168 leader_election.cc:288] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [CANDIDATE]: Term 1 election: Requested vote from peers 095a234565e84f9881c132fe2d1703d0 (127.18.193.3:45895), f074f016930e466884813151992106e9 (127.18.193.2:46863)
I20260807 04:14:38.082211 7092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6d7e9c7dba4d8ea9601227a3e51ebe" candidate_uuid: "d323c21e9535447695ff8981d6822dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095a234565e84f9881c132fe2d1703d0"
I20260807 04:14:38.082280 6946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6d7e9c7dba4d8ea9601227a3e51ebe" candidate_uuid: "d323c21e9535447695ff8981d6822dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f074f016930e466884813151992106e9"
I20260807 04:14:38.082453 7092 raft_consensus.cc:3037] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:38.082530 6946 raft_consensus.cc:3037] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:38.084837 7092 raft_consensus.cc:2454] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d323c21e9535447695ff8981d6822dee in term 1.
I20260807 04:14:38.084888 6946 raft_consensus.cc:2454] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d323c21e9535447695ff8981d6822dee in term 1.
I20260807 04:14:38.085312 6729 leader_election.cc:302] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d323c21e9535447695ff8981d6822dee, f074f016930e466884813151992106e9; no voters:
I20260807 04:14:38.085752 7168 raft_consensus.cc:2781] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:38.086252 7168 raft_consensus.cc:686] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 1 LEADER]: Becoming Leader. State: Replica: d323c21e9535447695ff8981d6822dee, State: Running, Role: LEADER
I20260807 04:14:38.086798 7168 consensus_queue.cc:237] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } }
I20260807 04:14:38.091642 6646 catalog_manager.cc:5495] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee reported cstate change: term changed from 0 to 1, leader changed from <none> to d323c21e9535447695ff8981d6822dee (127.18.193.1). New cstate: current_term: 1 leader_uuid: "d323c21e9535447695ff8981d6822dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d323c21e9535447695ff8981d6822dee" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 35155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 } health_report { overall_health: UNKNOWN } } }
W20260807 04:14:38.107649 7144 tablet.cc:2369] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:38.119303 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.113552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46462 (local address 127.18.193.1:35155)
0807 04:14:38.113871 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:38.113880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:38.113903 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:14:38.114173 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:38.114178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:38.114278 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:38.114441 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:14:38.114849 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.115581 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.116998 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.117415 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.117569 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.118163 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:14:38.118669 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:14:38.118861 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:38.119026 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 04:14:38.119070 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260807 04:14:38.127779 6946 raft_consensus.cc:1261] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 1 FOLLOWER]: Refusing update from remote peer d323c21e9535447695ff8981d6822dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:38.127902 7092 raft_consensus.cc:1261] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 1 FOLLOWER]: Refusing update from remote peer d323c21e9535447695ff8981d6822dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:38.128886 7174 consensus_queue.cc:1035] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "f074f016930e466884813151992106e9" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:38.129251 7168 consensus_queue.cc:1035] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "095a234565e84f9881c132fe2d1703d0" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 45895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:38.139644 7177 mvcc.cc:205] Tried to move back new op lower bound from 7315767616001167360 to 7315767615847485440. Current Snapshot: MvccSnapshot[applied={T|T < 7315767616001167360}]
I20260807 04:14:38.149375 7179 mvcc.cc:205] Tried to move back new op lower bound from 7315767616001167360 to 7315767615847485440. Current Snapshot: MvccSnapshot[applied={T|T < 7315767616001167360}]
I20260807 04:14:38.156152 6646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:38.156454 6646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:38.160727 6646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60851e5e726848e3ac8013fad54ee341: Sending DeleteTablet for 3 replicas of tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe
I20260807 04:14:38.161986 6779 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:38 UTC) from {username='slave'} at 127.0.0.1:46458
I20260807 04:14:38.162214 7068 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:38 UTC) from {username='slave'} at 127.0.0.1:57742
I20260807 04:14:38.162949 7187 tablet_replica.cc:330] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: stopping tablet replica
I20260807 04:14:38.163416 7187 raft_consensus.cc:2229] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:14:38.163769 6924 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:38 UTC) from {username='slave'} at 127.0.0.1:51028
I20260807 04:14:38.164136 7187 raft_consensus.cc:2258] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:38.166163 7187 ts_tablet_manager.cc:1905] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:38.166281 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 6710
I20260807 04:14:38.174116 7187 ts_tablet_manager.cc:1918] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:38.174331 7187 log.cc:1196] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/4f6d7e9c7dba4d8ea9601227a3e51ebe
I20260807 04:14:38.174959 7187 ts_tablet_manager.cc:1939] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P d323c21e9535447695ff8981d6822dee: Deleting consensus metadata
I20260807 04:14:38.176946 6630 catalog_manager.cc:4841] TS d323c21e9535447695ff8981d6822dee (127.18.193.1:35155): tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe (table test-table [id=73395f93967446fdb878b73b051231d9]) successfully deleted
I20260807 04:14:38.185854 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.167410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.18.193.1:35155)
0807 04:14:38.167637 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:38.167645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:38.177155 (+ 9510us) server_negotiation.cc:407] Connection header received
0807 04:14:38.177434 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:38.177448 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:38.177553 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:38.177668 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:38.178305 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.179538 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.181581 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.182227 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.182325 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.183977 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:38.183990 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:38.184010 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:38.184893 (+ 883us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:38.184957 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:38.184985 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:38.185073 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:38.185327 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 04:14:38.185458 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:14:38.189355 6844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:38.197194 7189 tablet_replica.cc:330] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: stopping tablet replica
I20260807 04:14:38.197844 7188 tablet_replica.cc:330] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: stopping tablet replica
I20260807 04:14:38.198383 7188 raft_consensus.cc:2229] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:38.198982 7188 raft_consensus.cc:2258] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:38.199636 7189 raft_consensus.cc:2229] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:38.200148 7189 raft_consensus.cc:2258] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:38.201642 7188 ts_tablet_manager.cc:1905] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:38.202682 7189 ts_tablet_manager.cc:1905] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:38.208043 7188 ts_tablet_manager.cc:1918] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:38.208220 7188 log.cc:1196] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/4f6d7e9c7dba4d8ea9601227a3e51ebe
I20260807 04:14:38.208881 7188 ts_tablet_manager.cc:1939] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P f074f016930e466884813151992106e9: Deleting consensus metadata
I20260807 04:14:38.210655 7189 ts_tablet_manager.cc:1918] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:38.210808 7189 log.cc:1196] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/4f6d7e9c7dba4d8ea9601227a3e51ebe
I20260807 04:14:38.211024 6634 catalog_manager.cc:4841] TS f074f016930e466884813151992106e9 (127.18.193.2:46863): tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe (table test-table [id=73395f93967446fdb878b73b051231d9]) successfully deleted
I20260807 04:14:38.211297 7189 ts_tablet_manager.cc:1939] T 4f6d7e9c7dba4d8ea9601227a3e51ebe P 095a234565e84f9881c132fe2d1703d0: Deleting consensus metadata
I20260807 04:14:38.213009 6634 catalog_manager.cc:4841] TS 095a234565e84f9881c132fe2d1703d0 (127.18.193.3:45895): tablet 4f6d7e9c7dba4d8ea9601227a3e51ebe (table test-table [id=73395f93967446fdb878b73b051231d9]) successfully deleted
I20260807 04:14:38.294710 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 6710
I20260807 04:14:38.315631 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 6853
I20260807 04:14:38.322618 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.316441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.18.193.2:46863)
0807 04:14:38.316678 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:38.316684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:38.316704 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:14:38.316873 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:38.316878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:38.316985 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:38.317109 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:38.317494 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.318271 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.319561 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.319983 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.320108 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.321247 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:38.321257 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:38.321273 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:38.321854 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:38.321898 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:38.321918 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:38.322058 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:38.322183 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 04:14:38.322288 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:14:38.323292 6991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:38.412396 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 6853
I20260807 04:14:38.432391 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 7001
I20260807 04:14:38.439637 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:38.433271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.18.193.3:45895)
0807 04:14:38.433459 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:38.433467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:38.433553 (+ 86us) server_negotiation.cc:407] Connection header received
0807 04:14:38.433771 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:38.433777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:38.433878 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:38.433991 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:38.434410 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.435178 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.436455 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:38.436958 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:38.437100 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:38.438245 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:38.438262 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:38.438277 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:38.438865 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:38.438913 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:38.438943 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:38.439069 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:38.439203 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 04:14:38.439312 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:14:38.440392 7138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:38.529933 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 7001
I20260807 04:14:38.548718 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 6612
I20260807 04:14:38.549866 6679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:38.659116 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 6612
2026-08-07T04:14:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2891 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:38 dist-test-slave-8n49 krb5kdc[7199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:38 dist-test-slave-8n49 krb5kdc[7199](info): set up 2 sockets
Aug 07 04:14:38 dist-test-slave-8n49 krb5kdc[7199](info): commencing operation
krb5kdc: starting...
W20260807 04:14:39.830034 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:40 dist-test-slave-8n49 krb5kdc[7199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:14:40.140347 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35857
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:35857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:40.483304 7212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:40.483747 7212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:40.483938 7212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:40.491420 7212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:40.491534 7212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:40.491581 7212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:40.491639 7212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:40.491683 7212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:40.501633 7212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:35857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:35857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:40.502918 7212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:40.504271 7212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:40.516395 7217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:40.516624 7219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:40.516574 7221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:40.517791 7212 server_base.cc:1034] running on GCE node
I20260807 04:14:40.518708 7212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:40.520401 7212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:40.521646 7212 hybrid_clock.cc:648] HybridClock initialized: now 1786076080521589 us; error 71 us; skew 500 ppm
I20260807 04:14:40.522007 7212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:40 dist-test-slave-8n49 krb5kdc[7199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076080, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:40.525287 7212 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:14:40.526885 7212 webserver.cc:460] Webserver started at http://127.18.193.62:43913/ using document root <none> and password file <none>
I20260807 04:14:40.527622 7212 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:40.527776 7212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:40.528115 7212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:40.531268 7212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "1998b87f077c4adcbe782ea4aa2977dc"
format_stamp: "Formatted at 2026-08-07 04:14:40 on dist-test-slave-8n49"
server_key: "58b728c905a037ce4f6612c122c0b13a"
server_key_iv: "950ee6308b37105dce21d76e135f85ff"
server_key_version: "encryptionkey@0"
I20260807 04:14:40.532049 7212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "1998b87f077c4adcbe782ea4aa2977dc"
format_stamp: "Formatted at 2026-08-07 04:14:40 on dist-test-slave-8n49"
server_key: "58b728c905a037ce4f6612c122c0b13a"
server_key_iv: "950ee6308b37105dce21d76e135f85ff"
server_key_version: "encryptionkey@0"
I20260807 04:14:40.537319 7212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:14:40.540983 7229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:40.542356 7212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:14:40.542555 7212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "1998b87f077c4adcbe782ea4aa2977dc"
format_stamp: "Formatted at 2026-08-07 04:14:40 on dist-test-slave-8n49"
server_key: "58b728c905a037ce4f6612c122c0b13a"
server_key_iv: "950ee6308b37105dce21d76e135f85ff"
server_key_version: "encryptionkey@0"
I20260807 04:14:40.542769 7212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:40.557130 7212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:40.561496 7212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:40.561815 7212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:40.581219 7212 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:35857
I20260807 04:14:40.581239 7284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:35857 every 8 connection(s)
I20260807 04:14:40.582664 7212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:40.586314 7285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:40.591387 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 7212
I20260807 04:14:40.591599 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:40.592049 19204 external_mini_cluster.cc:1428] Setting key 729d02e32f8a1de4654c38eb08ea9b10
I20260807 04:14:40.598657 7285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc: Bootstrap starting.
I20260807 04:14:40.602484 7285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:40.603518 7285 log.cc:824] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:40.606639 7285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc: No bootstrap required, opened a new log
Aug 07 04:14:40 dist-test-slave-8n49 krb5kdc[7199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:40.617357 7285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1998b87f077c4adcbe782ea4aa2977dc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35857 } }
I20260807 04:14:40.617835 7285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:40.617955 7285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1998b87f077c4adcbe782ea4aa2977dc, State: Initialized, Role: FOLLOWER
I20260807 04:14:40.618386 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:40.593959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.18.193.62:35857)
0807 04:14:40.594863 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:40.594879 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:40.594938 (+ 59us) server_negotiation.cc:407] Connection header received
0807 04:14:40.597471 (+ 2533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:40.597500 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:40.598007 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:40.598449 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:40.599992 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:40.602077 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:40.603612 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:40.604657 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:40.604800 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:40.609412 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:40.609428 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:40.609447 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:40.609492 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:40.613637 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:40.614373 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:40.614383 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:40.614400 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:40.614609 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:40.614988 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:40.614994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:40.614998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:40.615784 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:40.616086 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:40.616544 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 04:14:40.617015 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":360,"threads_started":1}
I20260807 04:14:40.618637 7285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1998b87f077c4adcbe782ea4aa2977dc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35857 } }
I20260807 04:14:40.618947 7285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:40.619110 7285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:40.619292 7285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:40.621927 7285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1998b87f077c4adcbe782ea4aa2977dc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35857 } }
I20260807 04:14:40.622413 7285 leader_election.cc:302] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1998b87f077c4adcbe782ea4aa2977dc; no voters:
I20260807 04:14:40.622994 7285 leader_election.cc:288] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:40.623123 7291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:40.623842 7291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [term 1 LEADER]: Becoming Leader. State: Replica: 1998b87f077c4adcbe782ea4aa2977dc, State: Running, Role: LEADER
I20260807 04:14:40.624394 7291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1998b87f077c4adcbe782ea4aa2977dc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35857 } }
I20260807 04:14:40.625165 7285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:40.627033 7292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1998b87f077c4adcbe782ea4aa2977dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1998b87f077c4adcbe782ea4aa2977dc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35857 } } }
I20260807 04:14:40.627365 7292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:40.628319 7293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1998b87f077c4adcbe782ea4aa2977dc. Latest consensus state: current_term: 1 leader_uuid: "1998b87f077c4adcbe782ea4aa2977dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1998b87f077c4adcbe782ea4aa2977dc" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 35857 } } }
I20260807 04:14:40.628726 7293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:40.629689 7298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:40.634790 7298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:40.645493 7298 catalog_manager.cc:1347] Generated new cluster ID: 3866824b08ee4e5b8030e5d1f6a0be06
I20260807 04:14:40.645617 7298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:40.654539 7298 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:40.655889 7298 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:40.670114 7298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc: Generated new TSK 0
I20260807 04:14:40.671041 7298 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.193.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
Entry for principal kudu/127.18.193.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab.
I20260807 04:14:40.740505 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35857
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:41.093855 7314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:41.094188 7314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:41.094368 7314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:41.101771 7314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:41.101922 7314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:41.102114 7314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:14:41.112149 7314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.1.keytab
--principal=kudu/127.18.193.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:41.113574 7314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:41.114969 7314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:41.128865 7322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:41.128865 7320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:41.129218 7319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:41.129784 7314 server_base.cc:1034] running on GCE node
I20260807 04:14:41.130357 7314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:41.131269 7314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:41.132498 7314 hybrid_clock.cc:648] HybridClock initialized: now 1786076081132423 us; error 74 us; skew 500 ppm
I20260807 04:14:41.132827 7314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:41 dist-test-slave-8n49 krb5kdc[7199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076081, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:41.135871 7314 init.cc:377] Logged in from keytab as kudu/127.18.193.1@KRBTEST.COM (short username kudu)
I20260807 04:14:41.137413 7314 webserver.cc:460] Webserver started at http://127.18.193.1:46159/ using document root <none> and password file <none>
I20260807 04:14:41.138038 7314 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:41.138140 7314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:41.138394 7314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:41.141258 7314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "f6e0ed08ddc7450f8872fa076a5897fd"
format_stamp: "Formatted at 2026-08-07 04:14:41 on dist-test-slave-8n49"
server_key: "acc4ceecf9093c43d626577913d25220"
server_key_iv: "12c75ba50dadd8e125cbdea4569aca05"
server_key_version: "encryptionkey@0"
I20260807 04:14:41.142045 7314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "f6e0ed08ddc7450f8872fa076a5897fd"
format_stamp: "Formatted at 2026-08-07 04:14:41 on dist-test-slave-8n49"
server_key: "acc4ceecf9093c43d626577913d25220"
server_key_iv: "12c75ba50dadd8e125cbdea4569aca05"
server_key_version: "encryptionkey@0"
I20260807 04:14:41.147730 7314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:14:41.151983 7331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:41.153360 7314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:41.153541 7314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "f6e0ed08ddc7450f8872fa076a5897fd"
format_stamp: "Formatted at 2026-08-07 04:14:41 on dist-test-slave-8n49"
server_key: "acc4ceecf9093c43d626577913d25220"
server_key_iv: "12c75ba50dadd8e125cbdea4569aca05"
server_key_version: "encryptionkey@0"
I20260807 04:14:41.153777 7314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:41.168519 7314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:41.173049 7314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:41.173353 7314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:41.174336 7314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:41.175688 7314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:41.175765 7314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:41.175918 7314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:41.176018 7314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:41.209337 7314 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:42649
I20260807 04:14:41.209383 7454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:42649 every 8 connection(s)
I20260807 04:14:41.210718 7314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:14:41.213003 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 7314
I20260807 04:14:41.213192 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:14:41.213681 19204 external_mini_cluster.cc:1428] Setting key 86eee4c6d3231669fc0c7d5339f8780a
Aug 07 04:14:41 dist-test-slave-8n49 krb5kdc[7199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076081, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.1@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:41.233392 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:41.213443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:58197 (local address 127.18.193.62:35857)
0807 04:14:41.213942 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:41.213948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:41.215118 (+ 1170us) server_negotiation.cc:407] Connection header received
0807 04:14:41.217297 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:41.217302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:41.217394 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:41.217521 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:41.219063 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.219793 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.221474 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.221878 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.222017 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:41.226118 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:41.226160 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:41.226164 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:41.226208 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:41.229951 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:41.230720 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:41.230728 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:41.230734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:41.230883 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:41.231312 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:41.231320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:41.231326 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:41.231730 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:41.231885 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:41.232653 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 04:14:41.232910 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":228,"threads_started":1}
I20260807 04:14:41.234105 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:41.214082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35857 (local address 127.18.193.1:58197)
0807 04:14:41.214925 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:41.214982 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:41.216971 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:41.217776 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:41.217784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:41.218305 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:41.218881 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:41.218898 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.219976 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.219980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:41.221317 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:41.221330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.222124 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.222131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:41.222356 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:41.223072 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:41.223095 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:41.225892 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:41.230113 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:41.230120 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:41.230133 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:41.230580 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:41.230991 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:41.230994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:41.230997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:41.231177 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:41.231856 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:41.231861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:41.232196 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 04:14:41.232436 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 04:14:41.232916 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":325,"threads_started":1}
I20260807 04:14:41.235725 7455 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35857
I20260807 04:14:41.236091 7455 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:41.237040 7455 heartbeater.cc:511] Master 127.18.193.62:35857 requested a full tablet report, sending...
I20260807 04:14:41.239316 7246 ts_manager.cc:194] Registered new tserver with Master: f6e0ed08ddc7450f8872fa076a5897fd (127.18.193.1:42649)
I20260807 04:14:41.240936 7246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.1@KRBTEST.COM'} at 127.18.193.1:58197
WARNING: no policy specified for kudu/127.18.193.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
Entry for principal kudu/127.18.193.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab.
I20260807 04:14:41.284215 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35857
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:41.632747 7462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:41.633049 7462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:41.633212 7462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:41.640794 7462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:41.640911 7462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:41.641129 7462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:14:41.651909 7462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.2.keytab
--principal=kudu/127.18.193.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:41.653422 7462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:41.654753 7462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:41.668715 7471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:41.668594 7469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:41.668895 7468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:41.669407 7462 server_base.cc:1034] running on GCE node
I20260807 04:14:41.670126 7462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:41.671212 7462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:41.672403 7462 hybrid_clock.cc:648] HybridClock initialized: now 1786076081672318 us; error 64 us; skew 500 ppm
I20260807 04:14:41.672752 7462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:41 dist-test-slave-8n49 krb5kdc[7199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076081, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:41.675945 7462 init.cc:377] Logged in from keytab as kudu/127.18.193.2@KRBTEST.COM (short username kudu)
I20260807 04:14:41.677556 7462 webserver.cc:460] Webserver started at http://127.18.193.2:43027/ using document root <none> and password file <none>
I20260807 04:14:41.678269 7462 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:41.678421 7462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:41.678750 7462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:41.681741 7462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "13475177e89246a39fbbfeca50755fa8"
format_stamp: "Formatted at 2026-08-07 04:14:41 on dist-test-slave-8n49"
server_key: "92cc51b0877a7f4a894a332e39a35185"
server_key_iv: "dd876d21e5bfe562a309d0885e2f3d6d"
server_key_version: "encryptionkey@0"
I20260807 04:14:41.682605 7462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "13475177e89246a39fbbfeca50755fa8"
format_stamp: "Formatted at 2026-08-07 04:14:41 on dist-test-slave-8n49"
server_key: "92cc51b0877a7f4a894a332e39a35185"
server_key_iv: "dd876d21e5bfe562a309d0885e2f3d6d"
server_key_version: "encryptionkey@0"
I20260807 04:14:41.688032 7462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 04:14:41.691663 7478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:41.693137 7462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:41.693327 7462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "13475177e89246a39fbbfeca50755fa8"
format_stamp: "Formatted at 2026-08-07 04:14:41 on dist-test-slave-8n49"
server_key: "92cc51b0877a7f4a894a332e39a35185"
server_key_iv: "dd876d21e5bfe562a309d0885e2f3d6d"
server_key_version: "encryptionkey@0"
I20260807 04:14:41.693574 7462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:41.707455 7462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:41.711900 7462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:41.712208 7462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:41.713478 7462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:41.714802 7462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:41.714910 7462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:41.715061 7462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:41.715143 7462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:41.751698 7462 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:36897
I20260807 04:14:41.751746 7596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:36897 every 8 connection(s)
I20260807 04:14:41.753101 7462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:14:41.754122 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 7462
I20260807 04:14:41.754309 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:14:41.754779 19204 external_mini_cluster.cc:1428] Setting key b8e67b9aad505560a360190413897baf
Aug 07 04:14:41 dist-test-slave-8n49 krb5kdc[7199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076081, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.2@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:41.775249 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:41.755985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:49927 (local address 127.18.193.62:35857)
0807 04:14:41.757426 (+ 1441us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:41.757433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:41.757457 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:14:41.760640 (+ 3183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:41.760647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:41.760748 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:41.760873 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:41.762412 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.763614 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.765244 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.765681 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.765818 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:41.769379 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:41.769391 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:41.769394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:41.769426 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:41.771997 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:41.772846 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:41.772850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:41.772852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:41.772978 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:41.773433 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:41.773437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:41.773439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:41.773791 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:41.773947 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:41.774632 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 04:14:41.774860 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313,"thread_start_us":1209,"threads_started":1}
I20260807 04:14:41.776398 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:41.756402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35857 (local address 127.18.193.2:49927)
0807 04:14:41.757301 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:41.757349 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:41.760315 (+ 2966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:41.761115 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:41.761124 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:41.761662 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:41.762229 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:41.762247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.763768 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.763774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:41.765095 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:41.765108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:41.765910 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:41.765918 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:41.766149 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:41.766879 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:41.766903 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:41.769146 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:41.772151 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:41.772161 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:41.772178 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:41.772715 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:41.773091 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:41.773096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:41.773099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:41.773297 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:41.773927 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:41.773934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:41.774276 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 04:14:41.774536 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 04:14:41.775084 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":386,"threads_started":1}
I20260807 04:14:41.778054 7598 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35857
I20260807 04:14:41.778378 7598 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:41.779111 7598 heartbeater.cc:511] Master 127.18.193.62:35857 requested a full tablet report, sending...
I20260807 04:14:41.780699 7246 ts_manager.cc:194] Registered new tserver with Master: 13475177e89246a39fbbfeca50755fa8 (127.18.193.2:36897)
I20260807 04:14:41.781963 7246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.2@KRBTEST.COM'} at 127.18.193.2:49927
WARNING: no policy specified for kudu/127.18.193.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
Entry for principal kudu/127.18.193.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab.
I20260807 04:14:41.820389 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35857
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:42.160766 7605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:42.161044 7605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:42.161242 7605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:42.168500 7605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:42.168689 7605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:42.168882 7605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:14:42.178906 7605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.3.keytab
--principal=kudu/127.18.193.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:35857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:42.180217 7605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:42.181528 7605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:42.194689 7611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:42.194736 7610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:42.194757 7613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:42.195545 7605 server_base.cc:1034] running on GCE node
I20260807 04:14:42.196091 7605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:42.197145 7605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:42.198345 7605 hybrid_clock.cc:648] HybridClock initialized: now 1786076082198301 us; error 34 us; skew 500 ppm
I20260807 04:14:42.198691 7605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:42 dist-test-slave-8n49 krb5kdc[7199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:42.201864 7605 init.cc:377] Logged in from keytab as kudu/127.18.193.3@KRBTEST.COM (short username kudu)
I20260807 04:14:42.203394 7605 webserver.cc:460] Webserver started at http://127.18.193.3:37289/ using document root <none> and password file <none>
I20260807 04:14:42.204092 7605 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:42.204200 7605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:42.204459 7605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:42.207372 7605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "f2f07a507b7f47508384cb4b8396c420"
format_stamp: "Formatted at 2026-08-07 04:14:42 on dist-test-slave-8n49"
server_key: "d503133d2d8a7a6062c9fbf789e7753f"
server_key_iv: "314496a254465b32c9b16401a563ffb0"
server_key_version: "encryptionkey@0"
I20260807 04:14:42.208110 7605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "f2f07a507b7f47508384cb4b8396c420"
format_stamp: "Formatted at 2026-08-07 04:14:42 on dist-test-slave-8n49"
server_key: "d503133d2d8a7a6062c9fbf789e7753f"
server_key_iv: "314496a254465b32c9b16401a563ffb0"
server_key_version: "encryptionkey@0"
I20260807 04:14:42.214756 7605 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 04:14:42.218369 7620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:42.219800 7605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:14:42.219978 7605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "f2f07a507b7f47508384cb4b8396c420"
format_stamp: "Formatted at 2026-08-07 04:14:42 on dist-test-slave-8n49"
server_key: "d503133d2d8a7a6062c9fbf789e7753f"
server_key_iv: "314496a254465b32c9b16401a563ffb0"
server_key_version: "encryptionkey@0"
I20260807 04:14:42.220185 7605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:42.232424 7605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:42.236897 7605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:42.237202 7605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:42.238173 7605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:42.239548 7605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:42.239624 7605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:42.239773 7605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:42.239822 7605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:42.244781 7455 heartbeater.cc:503] Master 127.18.193.62:35857 was elected leader, sending a full tablet report...
I20260807 04:14:42.275609 7605 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:44369
I20260807 04:14:42.275707 7740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:44369 every 8 connection(s)
I20260807 04:14:42.277100 7605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:14:42.280241 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 7605
I20260807 04:14:42.280465 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:14:42.281149 19204 external_mini_cluster.cc:1428] Setting key ff29391707a0504a48e3d1dda3cd5f15
Aug 07 04:14:42 dist-test-slave-8n49 krb5kdc[7199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.3@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:14:42.298539 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.279876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:57827 (local address 127.18.193.62:35857)
0807 04:14:42.280387 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.280395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.281404 (+ 1009us) server_negotiation.cc:407] Connection header received
0807 04:14:42.283738 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.283742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.283823 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.283925 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:42.285680 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.286510 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.288103 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.288626 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.288739 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.292290 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:42.292295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:42.292299 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:42.292332 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:42.295207 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.296193 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.296199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.296202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.296379 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.296939 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.296943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.296945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.297237 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:42.297371 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.298095 (+ 724us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.298261 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":244,"threads_started":1}
I20260807 04:14:42.299577 7749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.280284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:35857 (local address 127.18.193.3:57827)
0807 04:14:42.281203 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:42.281271 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:42.283395 (+ 2124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:42.284250 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:42.284263 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:42.284892 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:42.285490 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.285508 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.286682 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.286686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.287961 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.287978 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.288843 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.288848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.289082 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.289770 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:14:42.289793 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:42.292067 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:42.295374 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:42.295384 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:42.295434 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:42.296045 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:42.296498 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:14:42.296503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:14:42.296506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:14:42.296825 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:14:42.297391 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:42.297419 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:42.297750 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 04:14:42.297985 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 04:14:42.298473 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":367,"threads_started":1}
I20260807 04:14:42.301077 7742 heartbeater.cc:347] Connected to a master server at 127.18.193.62:35857
I20260807 04:14:42.301366 7742 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:42.302059 7742 heartbeater.cc:511] Master 127.18.193.62:35857 requested a full tablet report, sending...
I20260807 04:14:42.303455 7246 ts_manager.cc:194] Registered new tserver with Master: f2f07a507b7f47508384cb4b8396c420 (127.18.193.3:44369)
I20260807 04:14:42.304273 7246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.193.3@KRBTEST.COM'} at 127.18.193.3:57827
I20260807 04:14:42.309509 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:14:42.327526 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.313792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.18.193.62:35857)
0807 04:14:42.314089 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.314096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.314387 (+ 291us) server_negotiation.cc:407] Connection header received
0807 04:14:42.314526 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.314530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.314593 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.314679 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:42.316085 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.316898 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.318448 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.318866 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.318985 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.321581 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:42.321586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:42.321590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:42.321624 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:42.324424 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.325390 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.325395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.325399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.325531 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.326037 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.326041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.326044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.326517 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:42.326738 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.327084 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.327270 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 04:14:42.331913 7246 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:14:42.339350 7246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:14:42.365708 7762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.353197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:36897 (local address 127.0.0.1:35892)
0807 04:14:42.354200 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:42.354222 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:42.354369 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:42.355943 (+ 1574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:42.355955 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:42.356008 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:42.356509 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.356527 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.359146 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.359151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.362226 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.362242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.364675 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.364686 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.365019 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.365105 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 04:14:42.365227 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 04:14:42.365442 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"threads_started":1}
I20260807 04:14:42.367184 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.354362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35892 (local address 127.18.193.2:36897)
0807 04:14:42.355143 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.355156 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.355193 (+ 37us) server_negotiation.cc:407] Connection header received
0807 04:14:42.355318 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.355329 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.355646 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.356816 (+ 1170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:42.357088 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.358996 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.362473 (+ 3477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.364316 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.366306 (+ 1990us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.366557 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.366642 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.366846 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":813,"threads_started":1}
I20260807 04:14:42.371454 7527 tablet_service.cc:1467] Processing CreateTablet for tablet aefe16b4b27744338bb357273c276d1e (DEFAULT_TABLE table=test-table [id=094c6a31d4b74c23a8b2cc855c1eed1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:42.372979 7527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aefe16b4b27744338bb357273c276d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:42.373921 7762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.353571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42649 (local address 127.0.0.1:43060)
0807 04:14:42.365979 (+ 12408us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:42.365999 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:42.366106 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:42.367779 (+ 1673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:42.367785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:42.367811 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:42.368053 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.368065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.369775 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.369780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.371124 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.371139 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.373007 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.373017 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.373273 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.373308 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:14:42.373395 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 04:14:42.373553 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12294,"mutex_wait_us":62,"thread_start_us":1846,"threads_started":1}
I20260807 04:14:42.374574 7759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.352314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:44369 (local address 127.0.0.1:44986)
0807 04:14:42.353180 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:42.353232 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:42.353479 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:42.366265 (+ 12786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:42.366269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:42.366337 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:42.366761 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.366776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.368913 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.368921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.371105 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.371121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.373062 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.373068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.373608 (+ 540us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.373639 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:14:42.373732 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:14:42.373862 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":361,"thread_start_us":393,"threads_started":1}
I20260807 04:14:42.374817 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.352520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.18.193.3:44369)
0807 04:14:42.365547 (+ 13027us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.365559 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.365604 (+ 45us) server_negotiation.cc:407] Connection header received
0807 04:14:42.365727 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.365735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.366027 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.366278 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:42.366978 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.368646 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.371292 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.372809 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.373297 (+ 488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.373532 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.373817 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.374222 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12780,"thread_start_us":12518,"threads_started":1}
I20260807 04:14:42.374845 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.354988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.18.193.1:42649)
0807 04:14:42.365953 (+ 10965us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.365962 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.367095 (+ 1133us) server_negotiation.cc:407] Connection header received
0807 04:14:42.367304 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.367315 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.367561 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.368228 (+ 667us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:42.368479 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.369604 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.371349 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.372820 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.374019 (+ 1199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.374262 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.374333 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.374494 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10791,"thread_start_us":10976,"threads_started":1}
I20260807 04:14:42.378150 7671 tablet_service.cc:1467] Processing CreateTablet for tablet aefe16b4b27744338bb357273c276d1e (DEFAULT_TABLE table=test-table [id=094c6a31d4b74c23a8b2cc855c1eed1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:42.378537 7381 tablet_service.cc:1467] Processing CreateTablet for tablet aefe16b4b27744338bb357273c276d1e (DEFAULT_TABLE table=test-table [id=094c6a31d4b74c23a8b2cc855c1eed1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:42.379724 7671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aefe16b4b27744338bb357273c276d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:42.380020 7381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aefe16b4b27744338bb357273c276d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:42.389707 7765 tablet_bootstrap.cc:492] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Bootstrap starting.
I20260807 04:14:42.391710 7767 tablet_bootstrap.cc:492] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Bootstrap starting.
I20260807 04:14:42.393496 7765 tablet_bootstrap.cc:654] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:42.395181 7765 log.cc:824] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:42.395272 7767 tablet_bootstrap.cc:654] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:42.395274 7768 tablet_bootstrap.cc:492] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Bootstrap starting.
I20260807 04:14:42.396631 7767 log.cc:824] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:42.398869 7765 tablet_bootstrap.cc:492] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: No bootstrap required, opened a new log
I20260807 04:14:42.398942 7768 tablet_bootstrap.cc:654] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:42.399142 7765 ts_tablet_manager.cc:1397] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 04:14:42.399323 7767 tablet_bootstrap.cc:492] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: No bootstrap required, opened a new log
I20260807 04:14:42.399574 7767 ts_tablet_manager.cc:1397] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:14:42.400141 7768 log.cc:824] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:42.403080 7768 tablet_bootstrap.cc:492] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: No bootstrap required, opened a new log
I20260807 04:14:42.403337 7768 ts_tablet_manager.cc:1397] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 04:14:42.407758 7767 raft_consensus.cc:348] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.408151 7767 raft_consensus.cc:374] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:42.408255 7767 raft_consensus.cc:729] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f07a507b7f47508384cb4b8396c420, State: Initialized, Role: FOLLOWER
I20260807 04:14:42.408926 7767 consensus_queue.cc:260] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.409523 7765 raft_consensus.cc:348] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.409930 7765 raft_consensus.cc:374] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:42.410012 7765 raft_consensus.cc:729] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13475177e89246a39fbbfeca50755fa8, State: Initialized, Role: FOLLOWER
I20260807 04:14:42.410077 7742 heartbeater.cc:503] Master 127.18.193.62:35857 was elected leader, sending a full tablet report...
I20260807 04:14:42.410722 7767 ts_tablet_manager.cc:1428] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 04:14:42.410640 7765 consensus_queue.cc:260] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.411732 7598 heartbeater.cc:503] Master 127.18.193.62:35857 was elected leader, sending a full tablet report...
I20260807 04:14:42.413669 7765 ts_tablet_manager.cc:1428] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 04:14:42.414753 7768 raft_consensus.cc:348] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.415182 7768 raft_consensus.cc:374] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:42.415289 7768 raft_consensus.cc:729] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e0ed08ddc7450f8872fa076a5897fd, State: Initialized, Role: FOLLOWER
I20260807 04:14:42.415879 7768 consensus_queue.cc:260] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.417816 7768 ts_tablet_manager.cc:1428] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260807 04:14:42.462857 7773 raft_consensus.cc:482] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 04:14:42.463263 7456 tablet.cc:2369] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:42.463235 7773 raft_consensus.cc:504] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.465246 7773 leader_election.cc:288] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f07a507b7f47508384cb4b8396c420 (127.18.193.3:44369), f6e0ed08ddc7450f8872fa076a5897fd (127.18.193.1:42649)
I20260807 04:14:42.472623 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.465656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:42723 (local address 127.18.193.3:44369)
0807 04:14:42.465850 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.465854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.466564 (+ 710us) server_negotiation.cc:407] Connection header received
0807 04:14:42.466858 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.466864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.466946 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.467023 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:42.467823 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.468979 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.470831 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.471617 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.471809 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.471889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.472180 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.472315 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:14:42.472709 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.466699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:40065 (local address 127.18.193.1:42649)
0807 04:14:42.466931 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.466938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.467222 (+ 284us) server_negotiation.cc:407] Connection header received
0807 04:14:42.467470 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.467474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.467563 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.467657 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:42.468345 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.469219 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.470948 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.471717 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.471897 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.471983 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.472277 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.472421 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:14:42.472744 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.465663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:44369 (local address 127.18.193.2:42723)
0807 04:14:42.466446 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:42.466467 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:42.466621 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:42.467210 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:42.467213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:42.467234 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:42.467631 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.467647 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.469111 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.469119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.470657 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.470670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.471838 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.471845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.472007 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.472032 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:14:42.472150 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 04:14:42.472286 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":397,"threads_started":1}
I20260807 04:14:42.473162 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.466521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:42649 (local address 127.18.193.2:40065)
0807 04:14:42.467115 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:42.467129 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:42.467293 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:42.467687 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:42.467691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:42.467748 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:42.468161 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.468174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.469377 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.469381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.470786 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:42.470797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.471935 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.471941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:42.472116 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.472138 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 04:14:42.472249 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 04:14:42.472383 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":293,"threads_started":1}
I20260807 04:14:42.473821 7692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aefe16b4b27744338bb357273c276d1e" candidate_uuid: "13475177e89246a39fbbfeca50755fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f07a507b7f47508384cb4b8396c420" is_pre_election: true
I20260807 04:14:42.474066 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aefe16b4b27744338bb357273c276d1e" candidate_uuid: "13475177e89246a39fbbfeca50755fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" is_pre_election: true
I20260807 04:14:42.474386 7692 raft_consensus.cc:2454] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13475177e89246a39fbbfeca50755fa8 in term 0.
I20260807 04:14:42.474650 7403 raft_consensus.cc:2454] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13475177e89246a39fbbfeca50755fa8 in term 0.
I20260807 04:14:42.475137 7479 leader_election.cc:302] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13475177e89246a39fbbfeca50755fa8, f2f07a507b7f47508384cb4b8396c420; no voters:
I20260807 04:14:42.475508 7773 raft_consensus.cc:2781] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:14:42.475674 7773 raft_consensus.cc:482] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:14:42.475792 7773 raft_consensus.cc:3037] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:42.478247 7773 raft_consensus.cc:504] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.478957 7773 leader_election.cc:288] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [CANDIDATE]: Term 1 election: Requested vote from peers f2f07a507b7f47508384cb4b8396c420 (127.18.193.3:44369), f6e0ed08ddc7450f8872fa076a5897fd (127.18.193.1:42649)
I20260807 04:14:42.479391 7692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aefe16b4b27744338bb357273c276d1e" candidate_uuid: "13475177e89246a39fbbfeca50755fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f07a507b7f47508384cb4b8396c420"
I20260807 04:14:42.479496 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aefe16b4b27744338bb357273c276d1e" candidate_uuid: "13475177e89246a39fbbfeca50755fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e0ed08ddc7450f8872fa076a5897fd"
I20260807 04:14:42.479635 7692 raft_consensus.cc:3037] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:42.479741 7403 raft_consensus.cc:3037] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:42.482041 7692 raft_consensus.cc:2454] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13475177e89246a39fbbfeca50755fa8 in term 1.
I20260807 04:14:42.482098 7403 raft_consensus.cc:2454] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13475177e89246a39fbbfeca50755fa8 in term 1.
I20260807 04:14:42.482409 7479 leader_election.cc:302] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13475177e89246a39fbbfeca50755fa8, f2f07a507b7f47508384cb4b8396c420; no voters:
I20260807 04:14:42.482712 7773 raft_consensus.cc:2781] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:42.483219 7773 raft_consensus.cc:686] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 1 LEADER]: Becoming Leader. State: Replica: 13475177e89246a39fbbfeca50755fa8, State: Running, Role: LEADER
I20260807 04:14:42.483747 7773 consensus_queue.cc:237] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } }
I20260807 04:14:42.488309 7246 catalog_manager.cc:5495] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13475177e89246a39fbbfeca50755fa8 (127.18.193.2). New cstate: current_term: 1 leader_uuid: "13475177e89246a39fbbfeca50755fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13475177e89246a39fbbfeca50755fa8" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 36897 } health_report { overall_health: HEALTHY } } }
W20260807 04:14:42.505293 7599 tablet.cc:2369] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:42.521065 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.515945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.18.193.2:36897)
0807 04:14:42.516138 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.516143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.516160 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:14:42.516323 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.516327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.516404 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.516534 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:14:42.516915 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.517636 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.518986 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.519390 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.519495 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.519995 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:14:42.520507 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:14:42.520720 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.520781 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.520847 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 04:14:42.528923 7744 tablet.cc:2369] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:14:42.529608 7403 raft_consensus.cc:1261] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 1 FOLLOWER]: Refusing update from remote peer 13475177e89246a39fbbfeca50755fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:42.529672 7692 raft_consensus.cc:1261] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 1 FOLLOWER]: Refusing update from remote peer 13475177e89246a39fbbfeca50755fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:42.530706 7777 consensus_queue.cc:1035] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e0ed08ddc7450f8872fa076a5897fd" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 42649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:42.531141 7773 consensus_queue.cc:1035] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f07a507b7f47508384cb4b8396c420" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:42.544148 7781 mvcc.cc:205] Tried to move back new op lower bound from 7315767634030354432 to 7315767633858015232. Current Snapshot: MvccSnapshot[applied={T|T < 7315767634030354432 or (T in {7315767634030354432})}]
I20260807 04:14:42.544950 7783 mvcc.cc:205] Tried to move back new op lower bound from 7315767634030354432 to 7315767633858015232. Current Snapshot: MvccSnapshot[applied={T|T < 7315767634030354432}]
I20260807 04:14:42.556528 7246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:42.556856 7246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:42.560731 7246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1998b87f077c4adcbe782ea4aa2977dc: Sending DeleteTablet for 3 replicas of tablet aefe16b4b27744338bb357273c276d1e
I20260807 04:14:42.561939 7671 tablet_service.cc:1514] Processing DeleteTablet for tablet aefe16b4b27744338bb357273c276d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:42 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:44986
I20260807 04:14:42.562116 7527 tablet_service.cc:1514] Processing DeleteTablet for tablet aefe16b4b27744338bb357273c276d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:42 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:35892
I20260807 04:14:42.562124 7381 tablet_service.cc:1514] Processing DeleteTablet for tablet aefe16b4b27744338bb357273c276d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:42 UTC) from {username='kudu', principal='kudu/127.18.193.62@KRBTEST.COM'} at 127.0.0.1:43060
I20260807 04:14:42.564824 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 7314
I20260807 04:14:42.575002 7793 tablet_replica.cc:330] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: stopping tablet replica
I20260807 04:14:42.575767 7794 tablet_replica.cc:330] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: stopping tablet replica
I20260807 04:14:42.576946 7793 raft_consensus.cc:2229] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:14:42.577188 7794 raft_consensus.cc:2229] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:42.577432 7792 tablet_replica.cc:330] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: stopping tablet replica
I20260807 04:14:42.577813 7792 raft_consensus.cc:2229] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:42.578298 7792 raft_consensus.cc:2258] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:42.578526 7794 raft_consensus.cc:2258] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:42.578711 7793 raft_consensus.cc:2258] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:42.581079 7792 ts_tablet_manager.cc:1905] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:42.581801 7794 ts_tablet_manager.cc:1905] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:42.583058 7793 ts_tablet_manager.cc:1905] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:42.588039 7792 ts_tablet_manager.cc:1918] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:42.588215 7792 log.cc:1196] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/aefe16b4b27744338bb357273c276d1e
I20260807 04:14:42.588795 7792 ts_tablet_manager.cc:1939] T aefe16b4b27744338bb357273c276d1e P f2f07a507b7f47508384cb4b8396c420: Deleting consensus metadata
I20260807 04:14:42.589159 7794 ts_tablet_manager.cc:1918] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:42.589382 7794 log.cc:1196] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/aefe16b4b27744338bb357273c276d1e
I20260807 04:14:42.589836 7793 ts_tablet_manager.cc:1918] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:42.590001 7793 log.cc:1196] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/aefe16b4b27744338bb357273c276d1e
I20260807 04:14:42.590037 7794 ts_tablet_manager.cc:1939] T aefe16b4b27744338bb357273c276d1e P f6e0ed08ddc7450f8872fa076a5897fd: Deleting consensus metadata
I20260807 04:14:42.590489 7793 ts_tablet_manager.cc:1939] T aefe16b4b27744338bb357273c276d1e P 13475177e89246a39fbbfeca50755fa8: Deleting consensus metadata
I20260807 04:14:42.591205 7230 catalog_manager.cc:4841] TS f2f07a507b7f47508384cb4b8396c420 (127.18.193.3:44369): tablet aefe16b4b27744338bb357273c276d1e (table test-table [id=094c6a31d4b74c23a8b2cc855c1eed1a]) successfully deleted
I20260807 04:14:42.591923 7230 catalog_manager.cc:4841] TS f6e0ed08ddc7450f8872fa076a5897fd (127.18.193.1:42649): tablet aefe16b4b27744338bb357273c276d1e (table test-table [id=094c6a31d4b74c23a8b2cc855c1eed1a]) successfully deleted
I20260807 04:14:42.592022 7231 catalog_manager.cc:4841] TS 13475177e89246a39fbbfeca50755fa8 (127.18.193.2:36897): tablet aefe16b4b27744338bb357273c276d1e (table test-table [id=094c6a31d4b74c23a8b2cc855c1eed1a]) successfully deleted
Aug 07 04:14:42 dist-test-slave-8n49 krb5kdc[7199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.1@KRBTEST.COM
I20260807 04:14:42.609232 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.566247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.18.193.1:42649)
0807 04:14:42.566441 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.566449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.579242 (+ 12793us) server_negotiation.cc:407] Connection header received
0807 04:14:42.579594 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.579601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.579688 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.579800 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:42.592329 (+ 12529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.593573 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.595554 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.596216 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.596342 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.601943 (+ 5601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:42.601961 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:42.601980 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:42.602029 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:42.605774 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.606699 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.606706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.606709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.606880 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.607363 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.607368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.607370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.607829 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:42.608089 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.608549 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.608835 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:14:42.610020 7450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:42.701977 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 7314
I20260807 04:14:42.723449 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 7462
Aug 07 04:14:42 dist-test-slave-8n49 krb5kdc[7199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.2@KRBTEST.COM
I20260807 04:14:42.741341 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.724414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35916 (local address 127.18.193.2:36897)
0807 04:14:42.724643 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.724648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.724668 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:14:42.724842 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.724848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.724940 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.725083 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:42.726444 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.727288 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.728686 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.729148 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.729292 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.733963 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:42.733980 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:42.734000 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:42.734045 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:42.738116 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.738819 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.738829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.738833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.739040 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.739440 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.739445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.739448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.739922 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:42.740271 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.740665 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.740909 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:14:42.742211 7590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:42.859551 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 7462
I20260807 04:14:42.880149 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 7605
Aug 07 04:14:42 dist-test-slave-8n49 krb5kdc[7199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.3@KRBTEST.COM
I20260807 04:14:42.895928 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:42.881022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.18.193.3:44369)
0807 04:14:42.881244 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:42.881251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:42.881271 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:14:42.881506 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:42.881511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:42.881593 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:42.881704 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:42.883005 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.883743 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.885121 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:42.885534 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:42.885666 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:42.890263 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:42.890280 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:42.890296 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:42.890337 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:42.893102 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.893789 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.893796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.893799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.893947 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:42.894322 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:42.894327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:42.894330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:42.894701 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:42.895038 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:42.895344 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 04:14:42.895582 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 04:14:42.896759 7735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:42.987484 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 7605
I20260807 04:14:43.006394 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 7212
I20260807 04:14:43.007524 7278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:43.118378 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 7212
2026-08-07T04:14:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4461 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T04:14:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:43Z Disabled control of system clock
I20260807 04:14:43.209082 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:40183
--webserver_interface=127.18.193.62
--webserver_port=0
--builtin_ntp_servers=127.18.193.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:40183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:14:43.559185 7804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:43.559482 7804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:43.559564 7804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:43.566720 7804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:43.566818 7804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:43.566855 7804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:43.566913 7804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:43.566975 7804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:43.576040 7804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:40183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:40183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:43.577279 7804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:43.579115 7804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:43.590996 7809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:43.591061 7812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:43.591061 7810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:43.592222 7804 server_base.cc:1034] running on GCE node
I20260807 04:14:43.593111 7804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:43.594586 7804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:43.595817 7804 hybrid_clock.cc:648] HybridClock initialized: now 1786076083595778 us; error 64 us; skew 500 ppm
I20260807 04:14:43.596174 7804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:43.598593 7804 webserver.cc:460] Webserver started at http://127.18.193.62:35689/ using document root <none> and password file <none>
I20260807 04:14:43.599290 7804 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:43.599435 7804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:43.599776 7804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:43.602797 7804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "059195e7e78d406bbd0f78b44b4a04de"
format_stamp: "Formatted at 2026-08-07 04:14:43 on dist-test-slave-8n49"
server_key: "91e8b376d1900705eac01b6bd0f2f307"
server_key_iv: "ecec4eab4fc53f3be8e93181c47868a5"
server_key_version: "encryptionkey@0"
I20260807 04:14:43.603650 7804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "059195e7e78d406bbd0f78b44b4a04de"
format_stamp: "Formatted at 2026-08-07 04:14:43 on dist-test-slave-8n49"
server_key: "91e8b376d1900705eac01b6bd0f2f307"
server_key_iv: "ecec4eab4fc53f3be8e93181c47868a5"
server_key_version: "encryptionkey@0"
I20260807 04:14:43.608976 7804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:14:43.612651 7819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:43.614061 7804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:14:43.614266 7804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "059195e7e78d406bbd0f78b44b4a04de"
format_stamp: "Formatted at 2026-08-07 04:14:43 on dist-test-slave-8n49"
server_key: "91e8b376d1900705eac01b6bd0f2f307"
server_key_iv: "ecec4eab4fc53f3be8e93181c47868a5"
server_key_version: "encryptionkey@0"
I20260807 04:14:43.614526 7804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:43.641636 7804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:43.642571 7804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:43.642889 7804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:43.661917 7804 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:40183
I20260807 04:14:43.661937 7875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:40183 every 8 connection(s)
I20260807 04:14:43.663287 7804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:43.667191 7876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:43.667353 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 7804
I20260807 04:14:43.667543 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:43.668119 19204 external_mini_cluster.cc:1428] Setting key bbc2995cfbba2d2fc0ea3141fad8d92d
I20260807 04:14:43.679939 7876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de: Bootstrap starting.
I20260807 04:14:43.682286 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:43.670081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.18.193.62:40183)
0807 04:14:43.671122 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:43.671140 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:43.671182 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:14:43.673720 (+ 2538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:43.673744 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:43.674242 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:43.674685 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:43.675227 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:43.676460 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:43.677882 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:43.678531 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:43.678731 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:43.679956 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:43.679964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:43.679977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:43.680610 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:43.680700 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:43.680721 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:43.680843 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:43.681068 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 04:14:43.681372 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":372,"threads_started":1}
I20260807 04:14:43.683569 7876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:43.684712 7876 log.cc:824] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:43.687422 7876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de: No bootstrap required, opened a new log
I20260807 04:14:43.695015 7876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059195e7e78d406bbd0f78b44b4a04de" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40183 } }
I20260807 04:14:43.695364 7876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:43.695429 7876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059195e7e78d406bbd0f78b44b4a04de, State: Initialized, Role: FOLLOWER
I20260807 04:14:43.695911 7876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059195e7e78d406bbd0f78b44b4a04de" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40183 } }
I20260807 04:14:43.696187 7876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:43.696344 7876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:43.696614 7876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:43.698904 7876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059195e7e78d406bbd0f78b44b4a04de" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40183 } }
I20260807 04:14:43.699349 7876 leader_election.cc:302] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 059195e7e78d406bbd0f78b44b4a04de; no voters:
I20260807 04:14:43.699914 7876 leader_election.cc:288] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:43.700099 7881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:43.700898 7881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [term 1 LEADER]: Becoming Leader. State: Replica: 059195e7e78d406bbd0f78b44b4a04de, State: Running, Role: LEADER
I20260807 04:14:43.701427 7881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059195e7e78d406bbd0f78b44b4a04de" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40183 } }
I20260807 04:14:43.702097 7876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:43.704406 7883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 059195e7e78d406bbd0f78b44b4a04de. Latest consensus state: current_term: 1 leader_uuid: "059195e7e78d406bbd0f78b44b4a04de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059195e7e78d406bbd0f78b44b4a04de" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40183 } } }
I20260807 04:14:43.704852 7883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:43.704731 7882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "059195e7e78d406bbd0f78b44b4a04de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059195e7e78d406bbd0f78b44b4a04de" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40183 } } }
I20260807 04:14:43.705034 7882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:43.705818 7890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:43.711215 7890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:43.720582 7890 catalog_manager.cc:1347] Generated new cluster ID: c6cef3d972a44c2895e1a7efd32724d9
I20260807 04:14:43.720753 7890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:43.736129 7890 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:43.737457 7890 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:43.746055 7890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de: Generated new TSK 0
I20260807 04:14:43.747059 7890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:43.755573 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.1:0
--local_ip_for_outbound_sockets=127.18.193.1
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:40183
--builtin_ntp_servers=127.18.193.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:14:44.095996 7903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:44.096343 7903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:44.096513 7903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:44.104182 7903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:44.104327 7903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:44.104511 7903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.1
I20260807 04:14:44.114226 7903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.193.1
--webserver_port=0
--tserver_master_addrs=127.18.193.62:40183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.1
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:44.115633 7903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:44.117483 7903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:44.130697 7908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:44.130673 7911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:44.131021 7909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:44.132262 7903 server_base.cc:1034] running on GCE node
I20260807 04:14:44.133047 7903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:44.134116 7903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:44.135356 7903 hybrid_clock.cc:648] HybridClock initialized: now 1786076084135307 us; error 43 us; skew 500 ppm
I20260807 04:14:44.135726 7903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:44.142827 7903 webserver.cc:460] Webserver started at http://127.18.193.1:33707/ using document root <none> and password file <none>
I20260807 04:14:44.143623 7903 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:44.143762 7903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:44.144096 7903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:44.147143 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data/instance:
uuid: "2344f2a5742240139df5286b76863c42"
format_stamp: "Formatted at 2026-08-07 04:14:44 on dist-test-slave-8n49"
server_key: "f90672024bedfed5178e52915a2544ad"
server_key_iv: "2890866f83e305ee72214bcc561dcaa1"
server_key_version: "encryptionkey@0"
I20260807 04:14:44.147956 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance:
uuid: "2344f2a5742240139df5286b76863c42"
format_stamp: "Formatted at 2026-08-07 04:14:44 on dist-test-slave-8n49"
server_key: "f90672024bedfed5178e52915a2544ad"
server_key_iv: "2890866f83e305ee72214bcc561dcaa1"
server_key_version: "encryptionkey@0"
I20260807 04:14:44.153203 7903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:14:44.156916 7918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:44.158784 7903 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:14:44.158980 7903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal
uuid: "2344f2a5742240139df5286b76863c42"
format_stamp: "Formatted at 2026-08-07 04:14:44 on dist-test-slave-8n49"
server_key: "f90672024bedfed5178e52915a2544ad"
server_key_iv: "2890866f83e305ee72214bcc561dcaa1"
server_key_version: "encryptionkey@0"
I20260807 04:14:44.159245 7903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:44.176309 7903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:44.177219 7903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:44.177562 7903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:44.178601 7903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:44.180115 7903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:44.180251 7903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:44.180377 7903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:44.180471 7903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:44.215696 7903 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.1:44237
I20260807 04:14:44.215703 8039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.1:44237 every 8 connection(s)
I20260807 04:14:44.217130 7903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/data/info.pb
I20260807 04:14:44.226085 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 7903
I20260807 04:14:44.226287 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal/instance
I20260807 04:14:44.226768 19204 external_mini_cluster.cc:1428] Setting key d32c582861c7d4ff3da478bb700f6e87
I20260807 04:14:44.229821 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.2:0
--local_ip_for_outbound_sockets=127.18.193.2
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:40183
--builtin_ntp_servers=127.18.193.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:14:44.230580 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:44.220163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.1:46529 (local address 127.18.193.62:40183)
0807 04:14:44.220707 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:44.220711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:44.221717 (+ 1006us) server_negotiation.cc:407] Connection header received
0807 04:14:44.223622 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:44.223628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:44.223723 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:44.223845 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:44.224990 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.225740 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.227545 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.228053 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.228190 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:44.229646 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:44.229650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:44.229653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:44.229671 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:44.229711 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:44.229723 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:44.229823 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:44.230110 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 04:14:44.230218 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":268,"threads_started":1}
I20260807 04:14:44.231575 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:44.220509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:40183 (local address 127.18.193.1:46529)
0807 04:14:44.221501 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:44.221595 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:44.223345 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:44.224090 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:44.224098 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:44.224182 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:44.224812 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:44.224831 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.225907 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.225911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:44.227374 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:44.227393 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.228273 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.228278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:44.228477 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:44.229317 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:44.229320 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:44.229365 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:44.229441 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:44.229823 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:44.229837 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:44.229847 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:14:44.230076 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 04:14:44.230491 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":268,"threads_started":1}
I20260807 04:14:44.233187 8040 heartbeater.cc:347] Connected to a master server at 127.18.193.62:40183
I20260807 04:14:44.233505 8040 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:44.234256 8040 heartbeater.cc:511] Master 127.18.193.62:40183 requested a full tablet report, sending...
I20260807 04:14:44.236474 7839 ts_manager.cc:194] Registered new tserver with Master: 2344f2a5742240139df5286b76863c42 (127.18.193.1:44237)
I20260807 04:14:44.238713 7839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.1:46529
W20260807 04:14:44.575994 8044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:44.576270 8044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:44.576359 8044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:44.583702 8044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:44.583810 8044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:44.583917 8044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.2
I20260807 04:14:44.593143 8044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.193.2
--webserver_port=0
--tserver_master_addrs=127.18.193.62:40183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.2
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:44.594403 8044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:44.596124 8044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:44.609611 8050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:44.609680 8052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:44.609761 8049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:44.610651 8044 server_base.cc:1034] running on GCE node
I20260807 04:14:44.611267 8044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:44.612298 8044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:44.613575 8044 hybrid_clock.cc:648] HybridClock initialized: now 1786076084613527 us; error 51 us; skew 500 ppm
I20260807 04:14:44.613937 8044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:44.616251 8044 webserver.cc:460] Webserver started at http://127.18.193.2:37319/ using document root <none> and password file <none>
I20260807 04:14:44.617038 8044 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:44.617165 8044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:44.617493 8044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:44.620543 8044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data/instance:
uuid: "cca3a18d7024484fa592370630f560ae"
format_stamp: "Formatted at 2026-08-07 04:14:44 on dist-test-slave-8n49"
server_key: "3f93f85543ad6cb8234e9bbedd930069"
server_key_iv: "640b8c1222aa41ed23c66a36c8bad25c"
server_key_version: "encryptionkey@0"
I20260807 04:14:44.621475 8044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance:
uuid: "cca3a18d7024484fa592370630f560ae"
format_stamp: "Formatted at 2026-08-07 04:14:44 on dist-test-slave-8n49"
server_key: "3f93f85543ad6cb8234e9bbedd930069"
server_key_iv: "640b8c1222aa41ed23c66a36c8bad25c"
server_key_version: "encryptionkey@0"
I20260807 04:14:44.627008 8044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:14:44.630784 8058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:44.632167 8044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:14:44.632360 8044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal
uuid: "cca3a18d7024484fa592370630f560ae"
format_stamp: "Formatted at 2026-08-07 04:14:44 on dist-test-slave-8n49"
server_key: "3f93f85543ad6cb8234e9bbedd930069"
server_key_iv: "640b8c1222aa41ed23c66a36c8bad25c"
server_key_version: "encryptionkey@0"
I20260807 04:14:44.632678 8044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:44.657078 8044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:44.658393 8044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:44.658893 8044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:44.659945 8044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:44.661720 8044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:44.661852 8044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:44.661993 8044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:44.662082 8044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:44.699743 8044 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.2:46703
I20260807 04:14:44.699842 8179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.2:46703 every 8 connection(s)
I20260807 04:14:44.701296 8044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/data/info.pb
I20260807 04:14:44.711692 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8044
I20260807 04:14:44.711915 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal/instance
I20260807 04:14:44.712503 19204 external_mini_cluster.cc:1428] Setting key 15b9d27f698746920964b194f7b92a43
I20260807 04:14:44.714190 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:44.703972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.2:42695 (local address 127.18.193.62:40183)
0807 04:14:44.704146 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:44.704151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:44.705575 (+ 1424us) server_negotiation.cc:407] Connection header received
0807 04:14:44.707652 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:44.707657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:44.707774 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:44.707889 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:44.708975 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.709672 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.711203 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.711629 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.711761 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:44.713408 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:44.713414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:44.713419 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:44.713439 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:44.713482 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:44.713498 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:44.713610 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:44.713858 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 04:14:44.713938 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:14:44.715374 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:44.704438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:40183 (local address 127.18.193.2:42695)
0807 04:14:44.705412 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:44.705447 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:44.707373 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:44.708127 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:44.708136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:44.708226 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:44.708808 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:44.708825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.709829 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.709834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:44.711055 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:44.711067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:44.711867 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:44.711875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:44.712173 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:44.712996 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:44.712999 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:44.713086 (+ 87us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:44.713201 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:44.713622 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:44.713634 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:44.713641 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:14:44.713827 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 04:14:44.714214 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":279,"threads_started":1}
I20260807 04:14:44.715706 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.193.3:0
--local_ip_for_outbound_sockets=127.18.193.3
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:40183
--builtin_ntp_servers=127.18.193.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:14:44.716945 8180 heartbeater.cc:347] Connected to a master server at 127.18.193.62:40183
I20260807 04:14:44.717248 8180 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:44.718045 8180 heartbeater.cc:511] Master 127.18.193.62:40183 requested a full tablet report, sending...
I20260807 04:14:44.719648 7839 ts_manager.cc:194] Registered new tserver with Master: cca3a18d7024484fa592370630f560ae (127.18.193.2:46703)
I20260807 04:14:44.720901 7839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.2:42695
W20260807 04:14:45.064574 8183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:45.064986 8183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:45.065155 8183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:45.075096 8183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:45.075218 8183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:45.075394 8183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.193.3
I20260807 04:14:45.085187 8183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.193.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.193.3
--webserver_port=0
--tserver_master_addrs=127.18.193.62:40183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.193.3
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:45.086591 8183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:45.088336 8183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:45.101519 8191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:45.101503 8189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:45.101874 8188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:45.102298 8183 server_base.cc:1034] running on GCE node
I20260807 04:14:45.102986 8183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:45.104008 8183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:45.105242 8183 hybrid_clock.cc:648] HybridClock initialized: now 1786076085105192 us; error 43 us; skew 500 ppm
I20260807 04:14:45.105571 8183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:14:45.107918 8183 webserver.cc:460] Webserver started at http://127.18.193.3:39851/ using document root <none> and password file <none>
I20260807 04:14:45.108690 8183 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:45.108821 8183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:45.109145 8183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:45.112084 8183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data/instance:
uuid: "26a5665e3f2a4c579032aa9f64454917"
format_stamp: "Formatted at 2026-08-07 04:14:45 on dist-test-slave-8n49"
server_key: "9de838fc57cebc57889090f9d3efa944"
server_key_iv: "d3490c985d7674882c4b86a29abf7e77"
server_key_version: "encryptionkey@0"
I20260807 04:14:45.112982 8183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance:
uuid: "26a5665e3f2a4c579032aa9f64454917"
format_stamp: "Formatted at 2026-08-07 04:14:45 on dist-test-slave-8n49"
server_key: "9de838fc57cebc57889090f9d3efa944"
server_key_iv: "d3490c985d7674882c4b86a29abf7e77"
server_key_version: "encryptionkey@0"
I20260807 04:14:45.118332 8183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:14:45.122084 8198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:45.123564 8183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:14:45.123734 8183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal
uuid: "26a5665e3f2a4c579032aa9f64454917"
format_stamp: "Formatted at 2026-08-07 04:14:45 on dist-test-slave-8n49"
server_key: "9de838fc57cebc57889090f9d3efa944"
server_key_iv: "d3490c985d7674882c4b86a29abf7e77"
server_key_version: "encryptionkey@0"
I20260807 04:14:45.123996 8183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:45.145877 8183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:45.146759 8183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:45.147089 8183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:45.148123 8183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:14:45.149675 8183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:14:45.149801 8183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:45.149935 8183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:14:45.150024 8183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:45.187937 8183 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.3:35651
I20260807 04:14:45.188025 8316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.3:35651 every 8 connection(s)
I20260807 04:14:45.189317 8183 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/data/info.pb
I20260807 04:14:45.198050 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8183
I20260807 04:14:45.198253 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal/instance
I20260807 04:14:45.198702 19204 external_mini_cluster.cc:1428] Setting key b7c212d67de4967da2babad3f9c5836e
I20260807 04:14:45.201908 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.191802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:56587 (local address 127.18.193.62:40183)
0807 04:14:45.192039 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.192046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.193430 (+ 1384us) server_negotiation.cc:407] Connection header received
0807 04:14:45.195198 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.195203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.195274 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.195366 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:45.196358 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.197188 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.198858 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.199387 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.199544 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.201062 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:45.201066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:45.201069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:45.201085 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:45.201119 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:45.201132 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:45.201201 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.201500 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.201616 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 04:14:45.202971 8319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.192325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:40183 (local address 127.18.193.3:56587)
0807 04:14:45.193252 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:45.193292 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:45.194937 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:45.195557 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:45.195565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:45.195645 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:14:45.196200 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.196215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.197332 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.197336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.198682 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.198694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.199589 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.199594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.199904 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.200712 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:14:45.200716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:14:45.200770 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:14:45.200880 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:14:45.201231 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:14:45.201240 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:14:45.201248 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 04:14:45.201459 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 04:14:45.201872 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":322,"threads_started":1}
I20260807 04:14:45.204339 8317 heartbeater.cc:347] Connected to a master server at 127.18.193.62:40183
I20260807 04:14:45.204679 8317 heartbeater.cc:464] Registering TS with master...
I20260807 04:14:45.205463 8317 heartbeater.cc:511] Master 127.18.193.62:40183 requested a full tablet report, sending...
I20260807 04:14:45.206933 7839 ts_manager.cc:194] Registered new tserver with Master: 26a5665e3f2a4c579032aa9f64454917 (127.18.193.3:35651)
I20260807 04:14:45.207767 7839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.193.3:56587
I20260807 04:14:45.214046 19204 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:14:45.224825 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.218760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.18.193.62:40183)
0807 04:14:45.218965 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.218971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.219521 (+ 550us) server_negotiation.cc:407] Connection header received
0807 04:14:45.219729 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.219736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.219837 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.219940 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:45.220406 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.221264 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.222563 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.222946 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.223039 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.224040 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:45.224045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:45.224048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:45.224066 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:45.224099 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:45.224111 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:45.224211 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.224372 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.224467 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 04:14:45.228629 7839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:14:45.235409 7839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:14:45.241875 8040 heartbeater.cc:503] Master 127.18.193.62:40183 was elected leader, sending a full tablet report...
I20260807 04:14:45.258980 8331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.247869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.3:35651 (local address 127.0.0.1:46002)
0807 04:14:45.249499 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:45.249528 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:45.249735 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:45.251157 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:45.251171 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:45.251387 (+ 216us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:45.252392 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.252414 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.254319 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.254325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.256023 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.256035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.257986 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.257993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.258231 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.258283 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 04:14:45.258423 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 04:14:45.258634 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"thread_start_us":2486,"threads_started":1}
I20260807 04:14:45.259202 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.247943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.18.193.3:35651)
0807 04:14:45.249015 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.249021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.249647 (+ 626us) server_negotiation.cc:407] Connection header received
0807 04:14:45.249947 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.249955 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.250213 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.250429 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:45.252645 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.254198 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.256207 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.257775 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.257912 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.258191 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.258485 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.258864 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":758,"threads_started":1}
I20260807 04:14:45.260773 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.248210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:44237 (local address 127.0.0.1:49000)
0807 04:14:45.252139 (+ 3929us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:45.252164 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:45.252303 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:45.253035 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:45.253039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:45.253069 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:45.253585 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.253597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.255450 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.255456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.257285 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.257298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.259979 (+ 2681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.259986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.260156 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.260189 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:14:45.260277 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 04:14:45.260420 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3786,"thread_start_us":983,"threads_started":1}
I20260807 04:14:45.261091 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.250465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:46703 (local address 127.0.0.1:33936)
0807 04:14:45.252253 (+ 1788us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:45.252271 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:45.252439 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:45.253318 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:45.253323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:45.253347 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:45.253808 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.253820 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.256477 (+ 2657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.256483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.258532 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.258547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.260349 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.260355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.260530 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.260590 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 04:14:45.260682 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 04:14:45.260821 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1638,"thread_start_us":1290,"threads_started":1}
I20260807 04:14:45.261204 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.248287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49000 (local address 127.18.193.1:44237)
0807 04:14:45.250994 (+ 2707us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.251005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.252278 (+ 1273us) server_negotiation.cc:407] Connection header received
0807 04:14:45.252537 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.252566 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.252824 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.253030 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:45.253747 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.255301 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.257492 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.259796 (+ 2304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.259917 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.260181 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.260664 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.260842 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2506,"thread_start_us":2520,"threads_started":1}
I20260807 04:14:45.261801 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.250920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33936 (local address 127.18.193.2:46703)
0807 04:14:45.251515 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.251530 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.252359 (+ 829us) server_negotiation.cc:407] Connection header received
0807 04:14:45.252672 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.252711 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.252973 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.253245 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:45.253989 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.256326 (+ 2337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.258760 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.260170 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.260307 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.260517 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.261415 (+ 898us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.261551 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":505,"threads_started":1}
I20260807 04:14:45.263659 8249 tablet_service.cc:1467] Processing CreateTablet for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b (DEFAULT_TABLE table=test-table [id=12f5d440b4974c38869b123db54e7155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:45.264707 7969 tablet_service.cc:1467] Processing CreateTablet for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b (DEFAULT_TABLE table=test-table [id=12f5d440b4974c38869b123db54e7155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:45.265215 8249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:45.265240 8112 tablet_service.cc:1467] Processing CreateTablet for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b (DEFAULT_TABLE table=test-table [id=12f5d440b4974c38869b123db54e7155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:14:45.266112 7969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:45.266353 8112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:45.277657 8335 tablet_bootstrap.cc:492] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Bootstrap starting.
I20260807 04:14:45.281342 8336 tablet_bootstrap.cc:492] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Bootstrap starting.
I20260807 04:14:45.281983 8335 tablet_bootstrap.cc:654] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:45.283367 8335 log.cc:824] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:45.284796 8337 tablet_bootstrap.cc:492] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Bootstrap starting.
I20260807 04:14:45.284947 8336 tablet_bootstrap.cc:654] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:45.286317 8336 log.cc:824] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:45.288727 8337 tablet_bootstrap.cc:654] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:45.290103 8335 tablet_bootstrap.cc:492] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: No bootstrap required, opened a new log
I20260807 04:14:45.290199 8337 log.cc:824] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:45.290408 8335 ts_tablet_manager.cc:1397] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260807 04:14:45.292599 8336 tablet_bootstrap.cc:492] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: No bootstrap required, opened a new log
I20260807 04:14:45.292862 8336 ts_tablet_manager.cc:1397] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260807 04:14:45.293210 8337 tablet_bootstrap.cc:492] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: No bootstrap required, opened a new log
I20260807 04:14:45.293464 8337 ts_tablet_manager.cc:1397] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 04:14:45.300922 8337 raft_consensus.cc:348] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.301301 8337 raft_consensus.cc:374] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:45.301442 8337 raft_consensus.cc:729] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2344f2a5742240139df5286b76863c42, State: Initialized, Role: FOLLOWER
I20260807 04:14:45.302062 8337 consensus_queue.cc:260] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.301998 8335 raft_consensus.cc:348] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.302480 8335 raft_consensus.cc:374] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:45.302615 8335 raft_consensus.cc:729] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cca3a18d7024484fa592370630f560ae, State: Initialized, Role: FOLLOWER
I20260807 04:14:45.303035 8336 raft_consensus.cc:348] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.303476 8336 raft_consensus.cc:374] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:45.303274 8335 consensus_queue.cc:260] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.303601 8336 raft_consensus.cc:729] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a5665e3f2a4c579032aa9f64454917, State: Initialized, Role: FOLLOWER
I20260807 04:14:45.303905 8337 ts_tablet_manager.cc:1428] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260807 04:14:45.304423 8336 consensus_queue.cc:260] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.304976 8180 heartbeater.cc:503] Master 127.18.193.62:40183 was elected leader, sending a full tablet report...
I20260807 04:14:45.306275 8335 ts_tablet_manager.cc:1428] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260807 04:14:45.308588 8336 ts_tablet_manager.cc:1428] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260807 04:14:45.308674 8346 raft_consensus.cc:482] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:14:45.309094 8317 heartbeater.cc:503] Master 127.18.193.62:40183 was elected leader, sending a full tablet report...
I20260807 04:14:45.311244 8346 raft_consensus.cc:504] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.313617 8346 leader_election.cc:288] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2344f2a5742240139df5286b76863c42 (127.18.193.1:44237), cca3a18d7024484fa592370630f560ae (127.18.193.2:46703)
I20260807 04:14:45.321177 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.313983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:58033 (local address 127.18.193.2:46703)
0807 04:14:45.314208 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.314217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.314855 (+ 638us) server_negotiation.cc:407] Connection header received
0807 04:14:45.314954 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.314959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.315053 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.315158 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:45.315870 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.316930 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.319081 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.320266 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.320428 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.320507 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.320873 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.320977 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:14:45.321267 8319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.314458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.2:46703 (local address 127.18.193.3:58033)
0807 04:14:45.314659 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:45.314679 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:45.314827 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:45.315436 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:45.315447 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:45.315479 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:45.315699 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.315711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.317062 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.317067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.318918 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.318933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.320429 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.320434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.320692 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.320728 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:14:45.320837 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:14:45.320984 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":20}
I20260807 04:14:45.322114 8133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db17dcbd1d9e4a6f912b3909b4ae2d4b" candidate_uuid: "26a5665e3f2a4c579032aa9f64454917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cca3a18d7024484fa592370630f560ae" is_pre_election: true
I20260807 04:14:45.322640 8133 raft_consensus.cc:2454] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a5665e3f2a4c579032aa9f64454917 in term 0.
I20260807 04:14:45.322736 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.314041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.193.3:35135 (local address 127.18.193.1:44237)
0807 04:14:45.314236 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.314242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.315689 (+ 1447us) server_negotiation.cc:407] Connection header received
0807 04:14:45.315996 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.316001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.316086 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.316175 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:14:45.317034 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.318093 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.320681 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.321830 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.322010 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.322078 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.322332 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.322457 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:14:45.322814 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.314943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.1:44237 (local address 127.18.193.3:35135)
0807 04:14:45.315581 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 04:14:45.315600 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:14:45.315744 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:14:45.316323 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:14:45.316327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:14:45.316356 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:14:45.316861 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.316871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.318299 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.318304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.320474 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:14:45.320488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.321995 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.322002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:14:45.322175 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.322200 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:14:45.322306 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 04:14:45.322434 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":4,"thread_start_us":315,"threads_started":1}
I20260807 04:14:45.323343 8199 leader_election.cc:302] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a5665e3f2a4c579032aa9f64454917, cca3a18d7024484fa592370630f560ae; no voters:
I20260807 04:14:45.323720 8346 raft_consensus.cc:2781] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:14:45.323873 8346 raft_consensus.cc:482] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:14:45.324088 8346 raft_consensus.cc:3037] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:45.324425 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db17dcbd1d9e4a6f912b3909b4ae2d4b" candidate_uuid: "26a5665e3f2a4c579032aa9f64454917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2344f2a5742240139df5286b76863c42" is_pre_election: true
I20260807 04:14:45.324977 7991 raft_consensus.cc:2454] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a5665e3f2a4c579032aa9f64454917 in term 0.
I20260807 04:14:45.326908 8346 raft_consensus.cc:504] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.327647 8346 leader_election.cc:288] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [CANDIDATE]: Term 1 election: Requested vote from peers 2344f2a5742240139df5286b76863c42 (127.18.193.1:44237), cca3a18d7024484fa592370630f560ae (127.18.193.2:46703)
I20260807 04:14:45.328040 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db17dcbd1d9e4a6f912b3909b4ae2d4b" candidate_uuid: "26a5665e3f2a4c579032aa9f64454917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2344f2a5742240139df5286b76863c42"
I20260807 04:14:45.328261 7991 raft_consensus.cc:3037] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:45.328235 8133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db17dcbd1d9e4a6f912b3909b4ae2d4b" candidate_uuid: "26a5665e3f2a4c579032aa9f64454917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cca3a18d7024484fa592370630f560ae"
I20260807 04:14:45.328442 8133 raft_consensus.cc:3037] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:45.330655 7991 raft_consensus.cc:2454] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a5665e3f2a4c579032aa9f64454917 in term 1.
I20260807 04:14:45.330862 8133 raft_consensus.cc:2454] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a5665e3f2a4c579032aa9f64454917 in term 1.
I20260807 04:14:45.331046 8202 leader_election.cc:302] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2344f2a5742240139df5286b76863c42, 26a5665e3f2a4c579032aa9f64454917; no voters:
I20260807 04:14:45.331374 8346 raft_consensus.cc:2781] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:45.331574 8346 raft_consensus.cc:686] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 1 LEADER]: Becoming Leader. State: Replica: 26a5665e3f2a4c579032aa9f64454917, State: Running, Role: LEADER
I20260807 04:14:45.332052 8346 consensus_queue.cc:237] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } }
I20260807 04:14:45.336495 7838 catalog_manager.cc:5495] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26a5665e3f2a4c579032aa9f64454917 (127.18.193.3). New cstate: current_term: 1 leader_uuid: "26a5665e3f2a4c579032aa9f64454917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a5665e3f2a4c579032aa9f64454917" member_type: VOTER last_known_addr { host: "127.18.193.3" port: 35651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 } health_report { overall_health: UNKNOWN } } }
I20260807 04:14:45.368250 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.362624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46008 (local address 127.18.193.3:35651)
0807 04:14:45.362850 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.362856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.362875 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:14:45.363115 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.363119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.363198 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.363355 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:14:45.363807 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.364629 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.366105 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.366526 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.366661 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.367168 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:14:45.367676 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:14:45.367854 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.367913 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.367988 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:14:45.378854 8133 raft_consensus.cc:1261] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 1 FOLLOWER]: Refusing update from remote peer 26a5665e3f2a4c579032aa9f64454917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:45.378854 7991 raft_consensus.cc:1261] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 1 FOLLOWER]: Refusing update from remote peer 26a5665e3f2a4c579032aa9f64454917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:14:45.379783 8346 consensus_queue.cc:1035] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2344f2a5742240139df5286b76863c42" member_type: VOTER last_known_addr { host: "127.18.193.1" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:45.380102 8344 consensus_queue.cc:1035] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cca3a18d7024484fa592370630f560ae" member_type: VOTER last_known_addr { host: "127.18.193.2" port: 46703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:14:45.392299 8352 mvcc.cc:205] Tried to move back new op lower bound from 7315767645699260416 to 7315767645524353024. Current Snapshot: MvccSnapshot[applied={T|T < 7315767645699260416}]
I20260807 04:14:45.403667 7839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:45.403967 7839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:14:45.408051 7839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 059195e7e78d406bbd0f78b44b4a04de: Sending DeleteTablet for 3 replicas of tablet db17dcbd1d9e4a6f912b3909b4ae2d4b
I20260807 04:14:45.409308 8249 tablet_service.cc:1514] Processing DeleteTablet for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:45 UTC) from {username='slave'} at 127.0.0.1:46002
I20260807 04:14:45.409399 7969 tablet_service.cc:1514] Processing DeleteTablet for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:45 UTC) from {username='slave'} at 127.0.0.1:49000
I20260807 04:14:45.409763 8112 tablet_service.cc:1514] Processing DeleteTablet for tablet db17dcbd1d9e4a6f912b3909b4ae2d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:14:45 UTC) from {username='slave'} at 127.0.0.1:33936
I20260807 04:14:45.412043 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 7903
I20260807 04:14:45.421587 8360 tablet_replica.cc:330] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: stopping tablet replica
I20260807 04:14:45.422168 8360 raft_consensus.cc:2229] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:45.422757 8360 raft_consensus.cc:2258] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:45.425683 8360 ts_tablet_manager.cc:1905] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:45.433482 8360 ts_tablet_manager.cc:1918] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:45.433847 8360 log.cc:1196] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-0/wal/wals/db17dcbd1d9e4a6f912b3909b4ae2d4b
I20260807 04:14:45.434566 8360 ts_tablet_manager.cc:1939] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 2344f2a5742240139df5286b76863c42: Deleting consensus metadata
I20260807 04:14:45.436836 7825 catalog_manager.cc:4841] TS 2344f2a5742240139df5286b76863c42 (127.18.193.1:44237): tablet db17dcbd1d9e4a6f912b3909b4ae2d4b (table test-table [id=12f5d440b4974c38869b123db54e7155]) successfully deleted
I20260807 04:14:45.439458 8361 tablet_replica.cc:330] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: stopping tablet replica
I20260807 04:14:45.440064 8361 raft_consensus.cc:2229] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:14:45.442306 8361 raft_consensus.cc:2258] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:45.443270 8359 tablet_replica.cc:330] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: stopping tablet replica
I20260807 04:14:45.443682 8359 raft_consensus.cc:2229] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:14:45.444350 8359 raft_consensus.cc:2258] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:14:45.444546 8361 ts_tablet_manager.cc:1905] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:45.446254 8359 ts_tablet_manager.cc:1905] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:14:45.447770 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.413678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.18.193.1:44237)
0807 04:14:45.413903 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.413911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.438391 (+ 24480us) server_negotiation.cc:407] Connection header received
0807 04:14:45.438663 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.438669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.438791 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.438917 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:45.439683 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.440845 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.443048 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.443719 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.443823 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.445847 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:45.445859 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:45.445876 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:45.446712 (+ 836us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:45.446776 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:45.446802 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:45.446911 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.447251 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.447377 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:14:45.449232 8034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:45.452879 8361 ts_tablet_manager.cc:1918] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:45.453049 8361 log.cc:1196] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-1/wal/wals/db17dcbd1d9e4a6f912b3909b4ae2d4b
I20260807 04:14:45.453715 8361 ts_tablet_manager.cc:1939] T db17dcbd1d9e4a6f912b3909b4ae2d4b P cca3a18d7024484fa592370630f560ae: Deleting consensus metadata
I20260807 04:14:45.455735 7822 catalog_manager.cc:4841] TS cca3a18d7024484fa592370630f560ae (127.18.193.2:46703): tablet db17dcbd1d9e4a6f912b3909b4ae2d4b (table test-table [id=12f5d440b4974c38869b123db54e7155]) successfully deleted
I20260807 04:14:45.457067 8359 ts_tablet_manager.cc:1918] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:14:45.457273 8359 log.cc:1196] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Deleting WAL directory at /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786075749366013-19204-0/minicluster-data/ts-2/wal/wals/db17dcbd1d9e4a6f912b3909b4ae2d4b
I20260807 04:14:45.457759 8359 ts_tablet_manager.cc:1939] T db17dcbd1d9e4a6f912b3909b4ae2d4b P 26a5665e3f2a4c579032aa9f64454917: Deleting consensus metadata
I20260807 04:14:45.459308 7823 catalog_manager.cc:4841] TS 26a5665e3f2a4c579032aa9f64454917 (127.18.193.3:35651): tablet db17dcbd1d9e4a6f912b3909b4ae2d4b (table test-table [id=12f5d440b4974c38869b123db54e7155]) successfully deleted
I20260807 04:14:45.561934 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 7903
I20260807 04:14:45.582588 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8044
I20260807 04:14:45.590500 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.583527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.18.193.2:46703)
0807 04:14:45.583709 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.583715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.583736 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:14:45.583822 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.583825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.583923 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.584014 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:45.584460 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.585280 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.587176 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.587580 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.587672 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.588984 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:45.588993 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:45.589014 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:45.589760 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:45.589807 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:45.589828 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:45.589922 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.590076 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.590163 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:14:45.591331 8175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:45.691877 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8044
I20260807 04:14:45.717976 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8183
I20260807 04:14:45.726605 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:45.718818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.18.193.3:35651)
0807 04:14:45.719055 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:45.719061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:45.719085 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:14:45.719291 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:45.719297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:45.719432 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:45.719613 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:45.720084 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.721309 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.722592 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:45.723299 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:45.723445 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:45.724677 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:45.724691 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:45.724717 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:45.725648 (+ 931us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:45.725721 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:45.725756 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:45.725874 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:45.726044 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 04:14:45.726147 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:14:45.727655 8312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:45.854602 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8183
I20260807 04:14:45.880244 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 7804
I20260807 04:14:45.881446 7870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:45.992645 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 7804
2026-08-07T04:14:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2876 ms)
[----------] 6 tests from EncryptionPolicyTest (22295 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:46 dist-test-slave-8n49 krb5kdc[8370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:46 dist-test-slave-8n49 krb5kdc[8370](info): set up 2 sockets
Aug 07 04:14:46 dist-test-slave-8n49 krb5kdc[8370](info): commencing operation
krb5kdc: starting...
W20260807 04:14:47.178722 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:47 dist-test-slave-8n49 krb5kdc[8370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:14:47.500803 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:47.848829 8383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:47.849176 8383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:47.849349 8383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:47.856792 8383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:47.856950 8383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:47.857056 8383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:47.857172 8383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:47.857259 8383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:47.867228 8383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:47.868659 8383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:47.870026 8383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:47.882128 8392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:47.882193 8390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:47.882143 8389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:47.886163 8383 server_base.cc:1034] running on GCE node
I20260807 04:14:47.887053 8383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:47.888545 8383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:47.889827 8383 hybrid_clock.cc:648] HybridClock initialized: now 1786076087889786 us; error 35 us; skew 500 ppm
I20260807 04:14:47.890235 8383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:47 dist-test-slave-8n49 krb5kdc[8370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:47.893745 8383 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:14:47.895474 8383 webserver.cc:460] Webserver started at http://127.0.0.1:41253/ using document root <none> and password file <none>
I20260807 04:14:47.896229 8383 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:47.896363 8383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:47.896756 8383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:47.899869 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "2396e8de37b54f5491ea3d4a0758d644"
format_stamp: "Formatted at 2026-08-07 04:14:47 on dist-test-slave-8n49"
server_key: "032834fd7d0371a8fa8645b19d112557"
server_key_iv: "d4fdba222fa6f3d42c4754c56bc044cc"
server_key_version: "encryptionkey@0"
I20260807 04:14:47.900760 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "2396e8de37b54f5491ea3d4a0758d644"
format_stamp: "Formatted at 2026-08-07 04:14:47 on dist-test-slave-8n49"
server_key: "032834fd7d0371a8fa8645b19d112557"
server_key_iv: "d4fdba222fa6f3d42c4754c56bc044cc"
server_key_version: "encryptionkey@0"
I20260807 04:14:47.906013 8383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:14:47.909722 8399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:47.911335 8383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:14:47.911540 8383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "2396e8de37b54f5491ea3d4a0758d644"
format_stamp: "Formatted at 2026-08-07 04:14:47 on dist-test-slave-8n49"
server_key: "032834fd7d0371a8fa8645b19d112557"
server_key_iv: "d4fdba222fa6f3d42c4754c56bc044cc"
server_key_version: "encryptionkey@0"
I20260807 04:14:47.911768 8383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:47.925089 8383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:47.929595 8383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:47.929903 8383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:47.949913 8383 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34741
I20260807 04:14:47.949936 8455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34741 every 8 connection(s)
I20260807 04:14:47.951598 8383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:47.955437 8456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:47.961465 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8383
I20260807 04:14:47.961702 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:47.962363 19204 external_mini_cluster.cc:1428] Setting key 29021ed757295b82d0ac6f9bb73b0f7d
I20260807 04:14:47.966912 8456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644: Bootstrap starting.
I20260807 04:14:47.970891 8456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:47.972196 8456 log.cc:824] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:47.975556 8456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644: No bootstrap required, opened a new log
Aug 07 04:14:47 dist-test-slave-8n49 krb5kdc[8370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:14:47.985507 8456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2396e8de37b54f5491ea3d4a0758d644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34741 } }
I20260807 04:14:47.986029 8456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:47.986140 8456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2396e8de37b54f5491ea3d4a0758d644, State: Initialized, Role: FOLLOWER
I20260807 04:14:47.986830 8456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2396e8de37b54f5491ea3d4a0758d644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34741 } }
I20260807 04:14:47.987169 8456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:47.987321 8456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:47.987521 8456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:47.989286 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:47.964593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40507 (local address 127.0.0.1:34741)
0807 04:14:47.965486 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:47.965502 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:47.965558 (+ 56us) server_negotiation.cc:407] Connection header received
0807 04:14:47.967956 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:47.967983 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:47.968485 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:47.969004 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:47.970301 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:47.972398 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:47.974191 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:47.975201 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:47.975375 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:47.979666 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:47.979680 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:47.979699 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:47.979741 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:47.984011 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:47.984994 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:47.985003 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:47.985011 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:47.985220 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:47.985675 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:47.985680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:47.985683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:47.986404 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:47.986738 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:47.987217 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 04:14:47.987846 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":369,"threads_started":1}
I20260807 04:14:47.990912 8456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2396e8de37b54f5491ea3d4a0758d644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34741 } }
I20260807 04:14:47.991434 8456 leader_election.cc:302] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2396e8de37b54f5491ea3d4a0758d644; no voters:
I20260807 04:14:47.992089 8456 leader_election.cc:288] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:47.992200 8461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:47.993537 8461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [term 1 LEADER]: Becoming Leader. State: Replica: 2396e8de37b54f5491ea3d4a0758d644, State: Running, Role: LEADER
I20260807 04:14:47.993783 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:14:47.994114 8461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2396e8de37b54f5491ea3d4a0758d644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34741 } }
I20260807 04:14:47.994517 8456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:47.997022 8463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2396e8de37b54f5491ea3d4a0758d644. Latest consensus state: current_term: 1 leader_uuid: "2396e8de37b54f5491ea3d4a0758d644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2396e8de37b54f5491ea3d4a0758d644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34741 } } }
I20260807 04:14:47.997311 8462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2396e8de37b54f5491ea3d4a0758d644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2396e8de37b54f5491ea3d4a0758d644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34741 } } }
I20260807 04:14:47.997748 8462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:47.997607 8463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644 [sys.catalog]: This master's current role is: LEADER
W20260807 04:14:48.005805 8479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:14:48.005949 8479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:14:48.012938 8469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:48.014681 8469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:48.024735 8469 catalog_manager.cc:1347] Generated new cluster ID: 7afb95d6c635497194f6e0163fee0b27
I20260807 04:14:48.024878 8469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:48.031648 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:48.015177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32811 (local address 127.0.0.1:34741)
0807 04:14:48.015361 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:48.015368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:48.015419 (+ 51us) server_negotiation.cc:407] Connection header received
0807 04:14:48.015509 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:48.015514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:48.015589 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:48.015703 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:48.017872 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:48.018814 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:48.020115 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:48.020606 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:48.020749 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:48.023940 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:48.023947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:48.023954 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:48.024000 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:48.028086 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:48.029102 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:48.029110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:48.029119 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:48.029289 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:48.029799 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:48.029804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:48.029808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:48.030271 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:48.030619 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:48.030989 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 04:14:48.031229 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:14:48.041334 8469 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:48.043210 8469 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:48.051352 8469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2396e8de37b54f5491ea3d4a0758d644: Generated new TSK 0
I20260807 04:14:48.052402 8469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:48.062088 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8383
I20260807 04:14:48.063263 8450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:48.155316 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8383
W20260807 04:14:48.176393 8368 connection.cc:535] client connection to 127.0.0.1:34741 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34741): Connection reset by peer (error 104)
2026-08-07T04:14:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:48 dist-test-slave-8n49 krb5kdc[8490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:48 dist-test-slave-8n49 krb5kdc[8490](info): set up 2 sockets
Aug 07 04:14:48 dist-test-slave-8n49 krb5kdc[8490](info): commencing operation
krb5kdc: starting...
W20260807 04:14:49.330147 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:49 dist-test-slave-8n49 krb5kdc[8490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:14:49.635924 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41451
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:49.988010 8506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:49.988350 8506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:49.988516 8506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:49.995805 8506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:49.995970 8506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:49.996059 8506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:49.996148 8506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:49.996217 8506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:50.006189 8506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41451
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:50.007579 8506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:50.008927 8506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:50.020907 8512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:50.020942 8515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:50.020862 8513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:50.021737 8506 server_base.cc:1034] running on GCE node
I20260807 04:14:50.022459 8506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:50.023794 8506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:50.025032 8506 hybrid_clock.cc:648] HybridClock initialized: now 1786076090024991 us; error 65 us; skew 500 ppm
I20260807 04:14:50.025393 8506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:50 dist-test-slave-8n49 krb5kdc[8490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076090, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:50.032310 8506 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:14:50.033919 8506 webserver.cc:460] Webserver started at http://127.0.0.1:43713/ using document root <none> and password file <none>
I20260807 04:14:50.034626 8506 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:50.034780 8506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:50.035099 8506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:50.038267 8506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "eba439ba72664372be5f0c18e08fee6b"
format_stamp: "Formatted at 2026-08-07 04:14:50 on dist-test-slave-8n49"
server_key: "9035be5a50576629d8a83d6a542f381a"
server_key_iv: "455aa11373d2dd3b39651a1433b8bc6a"
server_key_version: "encryptionkey@0"
I20260807 04:14:50.039058 8506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "eba439ba72664372be5f0c18e08fee6b"
format_stamp: "Formatted at 2026-08-07 04:14:50 on dist-test-slave-8n49"
server_key: "9035be5a50576629d8a83d6a542f381a"
server_key_iv: "455aa11373d2dd3b39651a1433b8bc6a"
server_key_version: "encryptionkey@0"
I20260807 04:14:50.044267 8506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:14:50.047806 8522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:50.049230 8506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:50.049420 8506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "eba439ba72664372be5f0c18e08fee6b"
format_stamp: "Formatted at 2026-08-07 04:14:50 on dist-test-slave-8n49"
server_key: "9035be5a50576629d8a83d6a542f381a"
server_key_iv: "455aa11373d2dd3b39651a1433b8bc6a"
server_key_version: "encryptionkey@0"
I20260807 04:14:50.049628 8506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:50.082206 8506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:50.086516 8506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:50.086828 8506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:50.106029 8506 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41451
I20260807 04:14:50.106052 8580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41451 every 8 connection(s)
I20260807 04:14:50.107476 8506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:50.111263 8581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:50.117570 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8506
I20260807 04:14:50.117786 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:50.118233 19204 external_mini_cluster.cc:1428] Setting key ba1f94707a7d4c03f28217407e051230
I20260807 04:14:50.122154 8581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b: Bootstrap starting.
I20260807 04:14:50.125595 8581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:50.126629 8581 log.cc:824] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:50.129552 8581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b: No bootstrap required, opened a new log
I20260807 04:14:50.131668 8585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:50.120252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59435 (local address 127.0.0.1:41451)
0807 04:14:50.121114 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:50.121125 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:50.121167 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:14:50.122933 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:50.122956 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:50.123336 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:50.123655 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:50.124279 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:50.125600 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:50.127026 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:50.127720 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:50.127930 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:50.129226 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:50.129233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:50.129247 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:50.129835 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:50.129902 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:50.129929 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:50.130117 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:50.130388 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 04:14:50.130558 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":447,"threads_started":1}
I20260807 04:14:50.135601 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:14:50.137814 8581 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba439ba72664372be5f0c18e08fee6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41451 } }
I20260807 04:14:50.138203 8581 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:50.138330 8581 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba439ba72664372be5f0c18e08fee6b, State: Initialized, Role: FOLLOWER
I20260807 04:14:50.138999 8581 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba439ba72664372be5f0c18e08fee6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41451 } }
I20260807 04:14:50.139295 8581 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:50.139451 8581 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:50.139628 8581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:50.142834 8581 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba439ba72664372be5f0c18e08fee6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41451 } }
I20260807 04:14:50.143369 8581 leader_election.cc:302] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba439ba72664372be5f0c18e08fee6b; no voters:
I20260807 04:14:50.143978 8581 leader_election.cc:288] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:50.144114 8592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:50.144804 8592 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [term 1 LEADER]: Becoming Leader. State: Replica: eba439ba72664372be5f0c18e08fee6b, State: Running, Role: LEADER
I20260807 04:14:50.145382 8592 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba439ba72664372be5f0c18e08fee6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41451 } }
I20260807 04:14:50.148118 8585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:50.140214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42387 (local address 127.0.0.1:41451)
0807 04:14:50.140416 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:50.140423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:50.140835 (+ 412us) server_negotiation.cc:407] Connection header received
0807 04:14:50.140973 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:50.140979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:50.141060 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:50.141161 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:50.141672 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:50.142770 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:50.144500 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:50.145508 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:50.145675 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:50.147014 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:50.147023 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:50.147031 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:50.147059 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:50.147118 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:50.147145 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:50.147284 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:50.147567 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 04:14:50.147634 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:14:50.149952 8594 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba439ba72664372be5f0c18e08fee6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba439ba72664372be5f0c18e08fee6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41451 } } }
I20260807 04:14:50.150345 8594 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:50.150763 8593 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba439ba72664372be5f0c18e08fee6b. Latest consensus state: current_term: 1 leader_uuid: "eba439ba72664372be5f0c18e08fee6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba439ba72664372be5f0c18e08fee6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41451 } } }
I20260807 04:14:50.151170 8593 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:50.152393 8581 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:50.160893 8596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:50.166822 8596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:50.175652 8596 catalog_manager.cc:1347] Generated new cluster ID: ad5799d73a044473a3548bd442768213
I20260807 04:14:50.175786 8596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:50.205971 8596 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:50.207808 8596 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:50.217073 8596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba439ba72664372be5f0c18e08fee6b: Generated new TSK 0
I20260807 04:14:50.218140 8596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:50.236757 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8506
W20260807 04:14:50.237918 8576 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59435
W20260807 04:14:50.238754 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8506: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:14:50.238967 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8506
W20260807 04:14:50.260671 8488 connection.cc:535] client connection to 127.0.0.1:41451 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41451): Connection reset by peer (error 104)
2026-08-07T04:14:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2084 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:50 dist-test-slave-8n49 krb5kdc[8619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:50 dist-test-slave-8n49 krb5kdc[8619](info): set up 2 sockets
Aug 07 04:14:50 dist-test-slave-8n49 krb5kdc[8619](info): commencing operation
krb5kdc: starting...
W20260807 04:14:51.397156 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:51 dist-test-slave-8n49 krb5kdc[8619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:14:51.704540 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:52.044689 8632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:52.045009 8632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:52.045197 8632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:52.052733 8632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:52.052860 8632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:52.052992 8632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:52.053112 8632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:52.053195 8632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:52.063452 8632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:52.064922 8632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:52.066329 8632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:52.077632 8638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:52.078168 8639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:52.079372 8641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:52.080073 8632 server_base.cc:1034] running on GCE node
I20260807 04:14:52.080960 8632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:52.082257 8632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:52.083477 8632 hybrid_clock.cc:648] HybridClock initialized: now 1786076092083429 us; error 34 us; skew 500 ppm
I20260807 04:14:52.083796 8632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:52 dist-test-slave-8n49 krb5kdc[8619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076092, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:52.086918 8632 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:14:52.088478 8632 webserver.cc:460] Webserver started at http://127.0.0.1:44183/ using document root <none> and password file <none>
I20260807 04:14:52.089214 8632 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:52.089366 8632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:52.089679 8632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:52.092763 8632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "b414d6e6613344cfb5ad39092cf3ed2a"
format_stamp: "Formatted at 2026-08-07 04:14:52 on dist-test-slave-8n49"
server_key: "c34644daaf19e51002531944ae7d02ae"
server_key_iv: "ad9d7537f7ab5b24ead1c8c99d7ba083"
server_key_version: "encryptionkey@0"
I20260807 04:14:52.093569 8632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "b414d6e6613344cfb5ad39092cf3ed2a"
format_stamp: "Formatted at 2026-08-07 04:14:52 on dist-test-slave-8n49"
server_key: "c34644daaf19e51002531944ae7d02ae"
server_key_iv: "ad9d7537f7ab5b24ead1c8c99d7ba083"
server_key_version: "encryptionkey@0"
I20260807 04:14:52.098656 8632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:14:52.102157 8650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:52.103440 8632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:14:52.103648 8632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "b414d6e6613344cfb5ad39092cf3ed2a"
format_stamp: "Formatted at 2026-08-07 04:14:52 on dist-test-slave-8n49"
server_key: "c34644daaf19e51002531944ae7d02ae"
server_key_iv: "ad9d7537f7ab5b24ead1c8c99d7ba083"
server_key_version: "encryptionkey@0"
I20260807 04:14:52.103865 8632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:52.109843 8632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:52.114173 8632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:52.114492 8632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:52.135232 8632 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45475
I20260807 04:14:52.135288 8703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45475 every 8 connection(s)
I20260807 04:14:52.136859 8632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:52.141044 8704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:52.144238 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8632
I20260807 04:14:52.144445 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:52.145058 19204 external_mini_cluster.cc:1428] Setting key e96c6ef08533cf3a2879336e84572884
I20260807 04:14:52.153134 8707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:52.146971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36685 (local address 127.0.0.1:45475)
0807 04:14:52.147769 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:52.147781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:52.147822 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:14:52.149605 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:52.149628 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:52.149994 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:52.150408 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:52.151483 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:52.151489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:52.151501 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:52.151514 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:52.151567 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:52.151589 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:52.151829 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:52.151982 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 04:14:52.152158 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":362,"threads_started":1}
I20260807 04:14:52.153373 8704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a: Bootstrap starting.
I20260807 04:14:52.156684 8704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:52.156769 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:14:52.157732 8704 log.cc:824] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:52.160802 8704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a: No bootstrap required, opened a new log
I20260807 04:14:52.164083 8707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:52.161469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58389 (local address 127.0.0.1:45475)
0807 04:14:52.161656 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:52.161662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:52.162147 (+ 485us) server_negotiation.cc:407] Connection header received
0807 04:14:52.162277 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:52.162283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:52.162343 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:52.162455 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:52.163372 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:52.163383 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:52.163387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:52.163408 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:52.163458 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:52.163478 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:52.163605 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:52.163679 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 04:14:52.163737 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:14:52.169224 8704 raft_consensus.cc:348] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45475 } }
I20260807 04:14:52.169595 8704 raft_consensus.cc:374] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:52.169718 8704 raft_consensus.cc:729] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b414d6e6613344cfb5ad39092cf3ed2a, State: Initialized, Role: FOLLOWER
I20260807 04:14:52.170310 8704 consensus_queue.cc:260] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45475 } }
I20260807 04:14:52.170588 8704 raft_consensus.cc:388] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:52.170753 8704 raft_consensus.cc:482] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:52.170925 8704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:52.173187 8704 raft_consensus.cc:504] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45475 } }
I20260807 04:14:52.173666 8704 leader_election.cc:302] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b414d6e6613344cfb5ad39092cf3ed2a; no voters:
I20260807 04:14:52.174274 8704 leader_election.cc:288] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:52.174427 8714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:52.175120 8714 raft_consensus.cc:686] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [term 1 LEADER]: Becoming Leader. State: Replica: b414d6e6613344cfb5ad39092cf3ed2a, State: Running, Role: LEADER
I20260807 04:14:52.175673 8714 consensus_queue.cc:237] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45475 } }
I20260807 04:14:52.176301 8704 sys_catalog.cc:564] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:52.178709 8715 sys_catalog.cc:455] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45475 } } }
I20260807 04:14:52.179188 8715 sys_catalog.cc:458] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:52.180054 8716 sys_catalog.cc:455] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b414d6e6613344cfb5ad39092cf3ed2a. Latest consensus state: current_term: 1 leader_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b414d6e6613344cfb5ad39092cf3ed2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45475 } } }
I20260807 04:14:52.180480 8716 sys_catalog.cc:458] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:52.181123 8719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:52.187027 8719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:52.196969 8719 catalog_manager.cc:1347] Generated new cluster ID: 3ad50541b2a44146a783d00c22ecd7ae
I20260807 04:14:52.197088 8719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:52.224941 8719 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:52.225353 8719 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:52.231432 8719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b414d6e6613344cfb5ad39092cf3ed2a: Generated new TSK 0
I20260807 04:14:52.232134 8719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:52.236200 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8632
W20260807 04:14:52.237807 8699 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36685
W20260807 04:14:52.238514 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8632: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:14:52.238674 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8632
2026-08-07T04:14:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1997 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:52 dist-test-slave-8n49 krb5kdc[8736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:52 dist-test-slave-8n49 krb5kdc[8736](info): set up 2 sockets
Aug 07 04:14:52 dist-test-slave-8n49 krb5kdc[8736](info): commencing operation
krb5kdc: starting...
W20260807 04:14:53.396768 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:53 dist-test-slave-8n49 krb5kdc[8736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:14:53.713310 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:54.057819 8750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:54.058121 8750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:54.058296 8750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:54.065536 8750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:54.065651 8750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:54.065778 8750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:54.065896 8750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:54.065974 8750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:54.075850 8750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:54.077209 8750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:54.078501 8750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:54.090665 8759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:54.090725 8755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:54.090701 8756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:54.091559 8750 server_base.cc:1034] running on GCE node
I20260807 04:14:54.092352 8750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:54.093845 8750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:54.095093 8750 hybrid_clock.cc:648] HybridClock initialized: now 1786076094095045 us; error 54 us; skew 500 ppm
I20260807 04:14:54.095450 8750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:54 dist-test-slave-8n49 krb5kdc[8736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076094, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:54.098582 8750 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:14:54.100091 8750 webserver.cc:460] Webserver started at http://127.0.0.1:33513/ using document root <none> and password file <none>
I20260807 04:14:54.100868 8750 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:54.101018 8750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:54.101327 8750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:54.104323 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "e3e698cb48f9412a80186201a8959fea"
format_stamp: "Formatted at 2026-08-07 04:14:54 on dist-test-slave-8n49"
server_key: "dd15ff89dc77681fb8b11feeea2c96e5"
server_key_iv: "395c10749636b1bbab8dfa089a073079"
server_key_version: "encryptionkey@0"
I20260807 04:14:54.105165 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "e3e698cb48f9412a80186201a8959fea"
format_stamp: "Formatted at 2026-08-07 04:14:54 on dist-test-slave-8n49"
server_key: "dd15ff89dc77681fb8b11feeea2c96e5"
server_key_iv: "395c10749636b1bbab8dfa089a073079"
server_key_version: "encryptionkey@0"
I20260807 04:14:54.110277 8750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:14:54.113847 8767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:54.115132 8750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 04:14:54.115330 8750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "e3e698cb48f9412a80186201a8959fea"
format_stamp: "Formatted at 2026-08-07 04:14:54 on dist-test-slave-8n49"
server_key: "dd15ff89dc77681fb8b11feeea2c96e5"
server_key_iv: "395c10749636b1bbab8dfa089a073079"
server_key_version: "encryptionkey@0"
I20260807 04:14:54.115554 8750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:54.131346 8750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:54.135677 8750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:54.135984 8750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:54.155663 8750 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33027
I20260807 04:14:54.155694 8821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33027 every 8 connection(s)
I20260807 04:14:54.157110 8750 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:54.160930 8822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:54.164042 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8750
I20260807 04:14:54.164249 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:54.164825 19204 external_mini_cluster.cc:1428] Setting key f73fd5a3f65d4235929b35c4c006bccf
I20260807 04:14:54.172987 8822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea: Bootstrap starting.
I20260807 04:14:54.176111 8822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:54.177225 8822 log.cc:824] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:14:54 dist-test-slave-8n49 krb5kdc[8736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:14:54.180742 8822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea: No bootstrap required, opened a new log
I20260807 04:14:54.189172 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:54.166790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47529 (local address 127.0.0.1:33027)
0807 04:14:54.167611 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:54.167623 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:54.167662 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:14:54.169424 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:54.169446 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:54.169841 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:54.170255 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:54.171758 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:54.173208 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:54.174470 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:54.175135 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:54.175333 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:54.180626 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:54.180640 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:54.180664 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:54.180716 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:54.184506 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:54.185409 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:54.185415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:54.185421 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:54.185599 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:54.186113 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:54.186119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:54.186123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:54.186735 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:54.186980 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:54.187588 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 04:14:54.188046 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":388,"threads_started":1}
I20260807 04:14:54.191041 8822 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e698cb48f9412a80186201a8959fea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33027 } }
I20260807 04:14:54.191440 8822 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:54.191568 8822 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e698cb48f9412a80186201a8959fea, State: Initialized, Role: FOLLOWER
I20260807 04:14:54.192160 8822 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e698cb48f9412a80186201a8959fea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33027 } }
I20260807 04:14:54.192495 8822 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:54.192710 8822 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:54.192909 8822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:54.193238 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:14:54.196177 8822 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e698cb48f9412a80186201a8959fea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33027 } }
I20260807 04:14:54.196815 8822 leader_election.cc:302] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3e698cb48f9412a80186201a8959fea; no voters:
I20260807 04:14:54.200913 8822 leader_election.cc:288] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:54.201026 8830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:54.203795 8822 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:54.207711 8830 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [term 1 LEADER]: Becoming Leader. State: Replica: e3e698cb48f9412a80186201a8959fea, State: Running, Role: LEADER
I20260807 04:14:54.208539 8830 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e698cb48f9412a80186201a8959fea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33027 } }
I20260807 04:14:54.212098 8834 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3e698cb48f9412a80186201a8959fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e698cb48f9412a80186201a8959fea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33027 } } }
I20260807 04:14:54.212071 8833 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3e698cb48f9412a80186201a8959fea. Latest consensus state: current_term: 1 leader_uuid: "e3e698cb48f9412a80186201a8959fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e698cb48f9412a80186201a8959fea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33027 } } }
I20260807 04:14:54.212682 8834 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:54.212736 8833 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:54.214334 8844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:54.215497 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:54.198886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45709 (local address 127.0.0.1:33027)
0807 04:14:54.199048 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:54.199052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:54.199543 (+ 491us) server_negotiation.cc:407] Connection header received
0807 04:14:54.199665 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:54.199673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:54.199778 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:54.199911 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:54.201368 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:54.202507 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:54.203814 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:54.204347 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:54.204489 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:54.207137 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:54.207150 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:54.207153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:14:54.207182 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:54.211137 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:54.212110 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:54.212118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:54.212121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:54.212310 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:14:54.212896 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:14:54.212901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:14:54.212908 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:14:54.213431 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:54.214737 (+ 1306us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:54.214889 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 04:14:54.215104 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:14:54.220400 8844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:54.228896 8844 catalog_manager.cc:1347] Generated new cluster ID: 9708e887ff2142bb97ed2f31fe313cde
I20260807 04:14:54.229017 8844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:54.243053 8844 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:54.244691 8844 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:54.250104 8844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3e698cb48f9412a80186201a8959fea: Generated new TSK 0
I20260807 04:14:54.251127 8844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:54.257876 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8750
I20260807 04:14:54.259057 8817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:14:54.349190 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8750
2026-08-07T04:14:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2111 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:54 dist-test-slave-8n49 krb5kdc[8856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:54 dist-test-slave-8n49 krb5kdc[8856](info): set up 2 sockets
Aug 07 04:14:54 dist-test-slave-8n49 krb5kdc[8856](info): commencing operation
krb5kdc: starting...
W20260807 04:14:55.501947 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:55 dist-test-slave-8n49 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:14:55.816854 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:56.165771 8870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:56.166064 8870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:56.166240 8870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:56.173816 8870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:56.173946 8870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:56.174074 8870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:56.174191 8870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:56.174274 8870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:56.184255 8870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:56.185638 8870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:56.187026 8870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:56.199259 8875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:56.199213 8876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:56.199318 8880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:56.200173 8870 server_base.cc:1034] running on GCE node
I20260807 04:14:56.200977 8870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:56.202934 8870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:56.204166 8870 hybrid_clock.cc:648] HybridClock initialized: now 1786076096204101 us; error 58 us; skew 500 ppm
I20260807 04:14:56.204613 8870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:56 dist-test-slave-8n49 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:56.212313 8870 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:14:56.213902 8870 webserver.cc:460] Webserver started at http://127.0.0.1:45303/ using document root <none> and password file <none>
I20260807 04:14:56.214620 8870 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:56.214774 8870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:56.215102 8870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:56.218289 8870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "a552d92c43c04754b54d7d50603efa29"
format_stamp: "Formatted at 2026-08-07 04:14:56 on dist-test-slave-8n49"
server_key: "f04fa176db171062fa72573ca35c8eb1"
server_key_iv: "4e5c2ee2abeb325d2b0cb210fb243513"
server_key_version: "encryptionkey@0"
I20260807 04:14:56.219097 8870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "a552d92c43c04754b54d7d50603efa29"
format_stamp: "Formatted at 2026-08-07 04:14:56 on dist-test-slave-8n49"
server_key: "f04fa176db171062fa72573ca35c8eb1"
server_key_iv: "4e5c2ee2abeb325d2b0cb210fb243513"
server_key_version: "encryptionkey@0"
I20260807 04:14:56.224246 8870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 04:14:56.227764 8890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:56.229195 8870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:14:56.229393 8870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "a552d92c43c04754b54d7d50603efa29"
format_stamp: "Formatted at 2026-08-07 04:14:56 on dist-test-slave-8n49"
server_key: "f04fa176db171062fa72573ca35c8eb1"
server_key_iv: "4e5c2ee2abeb325d2b0cb210fb243513"
server_key_version: "encryptionkey@0"
I20260807 04:14:56.229612 8870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:56.242568 8870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:56.246819 8870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:56.247123 8870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:56.268725 8870 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33733
I20260807 04:14:56.268772 8945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33733 every 8 connection(s)
I20260807 04:14:56.270201 8870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:56.274058 8946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:56.277453 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8870
I20260807 04:14:56.277631 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:56.278174 19204 external_mini_cluster.cc:1428] Setting key da658b5cf13d3a48d0587d168976a49b
I20260807 04:14:56.286494 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29: Bootstrap starting.
I20260807 04:14:56.290237 8946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:56.291380 8946 log.cc:824] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:56.291947 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:56.280038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41005 (local address 127.0.0.1:33733)
0807 04:14:56.280894 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:56.280905 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:56.280944 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:14:56.282602 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:56.282625 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:56.283026 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:56.283346 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:56.283948 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:56.285388 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:56.287211 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:56.287872 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:56.288014 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:56.289378 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:56.289386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:56.289400 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:56.290003 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:56.290086 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:56.290116 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:56.290222 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:56.290540 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 04:14:56.290845 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":452,"threads_started":1}
I20260807 04:14:56.294751 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29: No bootstrap required, opened a new log
I20260807 04:14:56.295996 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:14:56.303478 8946 raft_consensus.cc:348] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a552d92c43c04754b54d7d50603efa29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 04:14:56.304003 8946 raft_consensus.cc:374] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:56.304121 8946 raft_consensus.cc:729] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a552d92c43c04754b54d7d50603efa29, State: Initialized, Role: FOLLOWER
I20260807 04:14:56.304867 8946 consensus_queue.cc:260] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a552d92c43c04754b54d7d50603efa29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 04:14:56.305156 8946 raft_consensus.cc:388] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:56.305305 8946 raft_consensus.cc:482] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:56.305531 8946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:56.308216 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:56.300931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46583 (local address 127.0.0.1:33733)
0807 04:14:56.301212 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:56.301224 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:56.301618 (+ 394us) server_negotiation.cc:407] Connection header received
0807 04:14:56.301810 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:56.301814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:56.301883 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:56.301977 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:56.302707 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:56.303492 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:56.305448 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:14:56.306039 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:14:56.306159 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:14:56.307373 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:56.307377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:56.307390 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:56.307409 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:56.307481 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:56.307500 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:56.307582 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:56.307757 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 04:14:56.307847 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:14:56.308815 8946 raft_consensus.cc:504] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a552d92c43c04754b54d7d50603efa29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 04:14:56.309448 8946 leader_election.cc:302] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a552d92c43c04754b54d7d50603efa29; no voters:
I20260807 04:14:56.310099 8946 leader_election.cc:288] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:56.310292 8956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:56.311080 8956 raft_consensus.cc:686] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [term 1 LEADER]: Becoming Leader. State: Replica: a552d92c43c04754b54d7d50603efa29, State: Running, Role: LEADER
I20260807 04:14:56.311708 8956 consensus_queue.cc:237] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a552d92c43c04754b54d7d50603efa29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 04:14:56.312335 8946 sys_catalog.cc:564] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:56.315140 8958 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a552d92c43c04754b54d7d50603efa29. Latest consensus state: current_term: 1 leader_uuid: "a552d92c43c04754b54d7d50603efa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a552d92c43c04754b54d7d50603efa29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260807 04:14:56.315603 8958 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:56.316174 8957 sys_catalog.cc:455] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a552d92c43c04754b54d7d50603efa29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a552d92c43c04754b54d7d50603efa29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260807 04:14:56.316534 8957 sys_catalog.cc:458] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29 [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:56.321887 8962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:56.326184 8962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:56.335870 8962 catalog_manager.cc:1347] Generated new cluster ID: 7e8f96a1f2364298b1c3bbc98282907a
I20260807 04:14:56.336009 8962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:56.371073 8962 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:56.373359 8962 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:56.390204 8962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a552d92c43c04754b54d7d50603efa29: Generated new TSK 0
I20260807 04:14:56.391351 8962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:56.408262 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8870
W20260807 04:14:56.409446 8941 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41005
W20260807 04:14:56.410229 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8870: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:14:56.410382 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8870
2026-08-07T04:14:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2060 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:56 dist-test-slave-8n49 krb5kdc[8979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:56 dist-test-slave-8n49 krb5kdc[8979](info): set up 2 sockets
Aug 07 04:14:56 dist-test-slave-8n49 krb5kdc[8979](info): commencing operation
krb5kdc: starting...
W20260807 04:14:57.574270 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:57 dist-test-slave-8n49 krb5kdc[8979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:14:57.880833 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:14:58.223153 8993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:14:58.223446 8993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:14:58.223626 8993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:14:58.230823 8993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:14:58.230937 8993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:14:58.231051 8993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:14:58.231153 8993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:14:58.231236 8993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:14:58.241218 8993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:14:58.242576 8993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:14:58.243876 8993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:14:58.256120 8998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:58.255991 8999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:14:58.256014 9001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:14:58.256794 8993 server_base.cc:1034] running on GCE node
I20260807 04:14:58.257728 8993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:14:58.259207 8993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:14:58.260504 8993 hybrid_clock.cc:648] HybridClock initialized: now 1786076098260448 us; error 61 us; skew 500 ppm
I20260807 04:14:58.260996 8993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:14:58 dist-test-slave-8n49 krb5kdc[8979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:14:58.264520 8993 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:14:58.266179 8993 webserver.cc:460] Webserver started at http://127.0.0.1:37447/ using document root <none> and password file <none>
I20260807 04:14:58.266855 8993 fs_manager.cc:359] Metadata directory not provided
I20260807 04:14:58.266973 8993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:14:58.267251 8993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:14:58.270645 8993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "969a7bf5af4540828e3c57deab6ea08d"
format_stamp: "Formatted at 2026-08-07 04:14:58 on dist-test-slave-8n49"
server_key: "db3cd78c30d0cc2ff474a6246a57fdfd"
server_key_iv: "2513c850b8f7fd97be16226b553931ad"
server_key_version: "encryptionkey@0"
I20260807 04:14:58.271476 8993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "969a7bf5af4540828e3c57deab6ea08d"
format_stamp: "Formatted at 2026-08-07 04:14:58 on dist-test-slave-8n49"
server_key: "db3cd78c30d0cc2ff474a6246a57fdfd"
server_key_iv: "2513c850b8f7fd97be16226b553931ad"
server_key_version: "encryptionkey@0"
I20260807 04:14:58.277066 8993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 04:14:58.281068 9009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:14:58.283198 8993 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:14:58.283427 8993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "969a7bf5af4540828e3c57deab6ea08d"
format_stamp: "Formatted at 2026-08-07 04:14:58 on dist-test-slave-8n49"
server_key: "db3cd78c30d0cc2ff474a6246a57fdfd"
server_key_iv: "2513c850b8f7fd97be16226b553931ad"
server_key_version: "encryptionkey@0"
I20260807 04:14:58.283638 8993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:14:58.291260 8993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:14:58.296437 8993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:14:58.296854 8993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:14:58.319375 8993 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37967
I20260807 04:14:58.319394 9062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37967 every 8 connection(s)
I20260807 04:14:58.321012 8993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:14:58.326365 9063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:14:58.330708 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 8993
I20260807 04:14:58.330901 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:14:58.331385 19204 external_mini_cluster.cc:1428] Setting key f116fda61afae605de5e8c0e407dd7d7
I20260807 04:14:58.341634 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:58.333432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56875 (local address 127.0.0.1:37967)
0807 04:14:58.334265 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:58.334307 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:58.334365 (+ 58us) server_negotiation.cc:407] Connection header received
0807 04:14:58.336763 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:58.336815 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:58.337248 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:58.337685 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:58.339116 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:58.339132 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:58.339148 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:58.339173 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:58.339292 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:58.339333 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:58.339636 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:58.339799 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 04:14:58.340053 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":324,"threads_started":1}
I20260807 04:14:58.342356 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d: Bootstrap starting.
I20260807 04:14:58.345676 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:14:58.346256 9063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d: Neither blocks nor log segments found. Creating new log.
I20260807 04:14:58.347330 9063 log.cc:824] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:14:58.350874 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d: No bootstrap required, opened a new log
I20260807 04:14:58.353886 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:14:58.351154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35129 (local address 127.0.0.1:37967)
0807 04:14:58.351331 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 04:14:58.351342 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:14:58.351822 (+ 480us) server_negotiation.cc:407] Connection header received
0807 04:14:58.351959 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:14:58.351964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:14:58.352028 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:14:58.352150 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:14:58.353187 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:14:58.353193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:14:58.353198 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:14:58.353216 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:14:58.353265 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:14:58.353290 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:14:58.353414 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 04:14:58.353499 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 04:14:58.353561 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:14:58.359308 9063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969a7bf5af4540828e3c57deab6ea08d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260807 04:14:58.359727 9063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:14:58.359836 9063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 969a7bf5af4540828e3c57deab6ea08d, State: Initialized, Role: FOLLOWER
I20260807 04:14:58.360414 9063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969a7bf5af4540828e3c57deab6ea08d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260807 04:14:58.360702 9063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:14:58.360857 9063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:14:58.361039 9063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:14:58.363232 9063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969a7bf5af4540828e3c57deab6ea08d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260807 04:14:58.363790 9063 leader_election.cc:302] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 969a7bf5af4540828e3c57deab6ea08d; no voters:
I20260807 04:14:58.364399 9063 leader_election.cc:288] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:14:58.364588 9074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:14:58.365321 9074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [term 1 LEADER]: Becoming Leader. State: Replica: 969a7bf5af4540828e3c57deab6ea08d, State: Running, Role: LEADER
I20260807 04:14:58.365938 9074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969a7bf5af4540828e3c57deab6ea08d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260807 04:14:58.366612 9063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:14:58.368940 9075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "969a7bf5af4540828e3c57deab6ea08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969a7bf5af4540828e3c57deab6ea08d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } } }
I20260807 04:14:58.369448 9075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:58.369840 9077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 969a7bf5af4540828e3c57deab6ea08d. Latest consensus state: current_term: 1 leader_uuid: "969a7bf5af4540828e3c57deab6ea08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969a7bf5af4540828e3c57deab6ea08d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } } }
I20260807 04:14:58.375711 9077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d [sys.catalog]: This master's current role is: LEADER
I20260807 04:14:58.377434 9085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:14:58.382992 9085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:14:58.392293 9085 catalog_manager.cc:1347] Generated new cluster ID: efef531a11764972b57cd57e14cad179
I20260807 04:14:58.392412 9085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:14:58.411509 9085 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:14:58.411959 9085 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:14:58.423805 9085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 969a7bf5af4540828e3c57deab6ea08d: Generated new TSK 0
I20260807 04:14:58.424700 9085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:14:58.430187 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8993
W20260807 04:14:58.431604 9057 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56875
W20260807 04:14:58.432271 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 8993: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:14:58.432426 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 8993
2026-08-07T04:14:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2019 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:14:58 dist-test-slave-8n49 krb5kdc[9098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:14:58 dist-test-slave-8n49 krb5kdc[9098](info): set up 2 sockets
Aug 07 04:14:58 dist-test-slave-8n49 krb5kdc[9098](info): commencing operation
krb5kdc: starting...
W20260807 04:14:59.583611 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:14:59 dist-test-slave-8n49 krb5kdc[9098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:14:59.885190 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35221
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:00.233201 9111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:00.233502 9111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:00.233600 9111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:00.240836 9111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:00.240954 9111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:00.241001 9111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:00.241056 9111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:00.241101 9111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:00.250919 9111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:00.252283 9111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:00.253865 9111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:00.265937 9116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:00.266000 9119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:00.265913 9117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:00.267107 9111 server_base.cc:1034] running on GCE node
I20260807 04:15:00.267854 9111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:00.269429 9111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:00.270628 9111 hybrid_clock.cc:648] HybridClock initialized: now 1786076100270582 us; error 41 us; skew 500 ppm
I20260807 04:15:00.270948 9111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:00 dist-test-slave-8n49 krb5kdc[9098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:00.274200 9111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:15:00.275730 9111 webserver.cc:460] Webserver started at http://127.0.0.1:40147/ using document root <none> and password file <none>
I20260807 04:15:00.276358 9111 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:00.276459 9111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:00.276805 9111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:00.279723 9111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "0d496445c46c471883788c1e64afb552"
format_stamp: "Formatted at 2026-08-07 04:15:00 on dist-test-slave-8n49"
server_key: "98fba5f5b079606abaf882576fc24e34"
server_key_iv: "8cf0e53b154b698715c75b69860003bf"
server_key_version: "encryptionkey@0"
I20260807 04:15:00.280450 9111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "0d496445c46c471883788c1e64afb552"
format_stamp: "Formatted at 2026-08-07 04:15:00 on dist-test-slave-8n49"
server_key: "98fba5f5b079606abaf882576fc24e34"
server_key_iv: "8cf0e53b154b698715c75b69860003bf"
server_key_version: "encryptionkey@0"
I20260807 04:15:00.285588 9111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:15:00.289115 9127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:00.290524 9111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:15:00.290714 9111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "0d496445c46c471883788c1e64afb552"
format_stamp: "Formatted at 2026-08-07 04:15:00 on dist-test-slave-8n49"
server_key: "98fba5f5b079606abaf882576fc24e34"
server_key_iv: "8cf0e53b154b698715c75b69860003bf"
server_key_version: "encryptionkey@0"
I20260807 04:15:00.290884 9111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:00.325466 9111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:00.329752 9111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:00.330070 9111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:00.349303 9111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35221
I20260807 04:15:00.349344 9181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35221 every 8 connection(s)
I20260807 04:15:00.350778 9111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:00.354890 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9111
I20260807 04:15:00.355109 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:00.355110 9182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:00.355754 19204 external_mini_cluster.cc:1428] Setting key b2d18fdf9a534a4090d2a87d45e8641e
I20260807 04:15:00.369444 9182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552: Bootstrap starting.
Aug 07 04:15:00 dist-test-slave-8n49 krb5kdc[9098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:15:00.373725 9182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:00.374925 9182 log.cc:824] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:00.378813 9182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552: No bootstrap required, opened a new log
I20260807 04:15:00.380122 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:00.357785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:55971 (local address 127.0.0.1:35221)
0807 04:15:00.358594 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:00.358607 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:00.358650 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:15:00.360490 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:00.360521 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:00.360981 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:00.361371 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:00.362860 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:00.364135 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:00.365497 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:00.366199 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:00.366441 (+ 242us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:00.371502 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:00.371512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:00.371527 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:00.371568 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:00.375037 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:00.376193 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:00.376204 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:00.376220 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:00.376523 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:00.377205 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:00.377209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:00.377211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:00.377788 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:00.378049 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:00.378625 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 04:15:00.379065 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":354,"threads_started":1}
I20260807 04:15:00.384284 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:00.389451 9182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d496445c46c471883788c1e64afb552" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35221 } }
I20260807 04:15:00.390013 9182 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:00.390170 9182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d496445c46c471883788c1e64afb552, State: Initialized, Role: FOLLOWER
I20260807 04:15:00.391042 9182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d496445c46c471883788c1e64afb552" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35221 } }
I20260807 04:15:00.391386 9182 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:00.391561 9182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:00.391757 9182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:00.395255 9182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d496445c46c471883788c1e64afb552" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35221 } }
I20260807 04:15:00.395922 9182 leader_election.cc:302] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d496445c46c471883788c1e64afb552; no voters:
I20260807 04:15:00.396721 9182 leader_election.cc:288] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:00.396847 9192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:00.399108 9182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:00.403635 9192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [term 1 LEADER]: Becoming Leader. State: Replica: 0d496445c46c471883788c1e64afb552, State: Running, Role: LEADER
I20260807 04:15:00.404312 9192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d496445c46c471883788c1e64afb552" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35221 } }
I20260807 04:15:00.407498 9193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d496445c46c471883788c1e64afb552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d496445c46c471883788c1e64afb552" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35221 } } }
I20260807 04:15:00.407961 9193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:00.408486 9194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d496445c46c471883788c1e64afb552. Latest consensus state: current_term: 1 leader_uuid: "0d496445c46c471883788c1e64afb552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d496445c46c471883788c1e64afb552" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35221 } } }
I20260807 04:15:00.408815 9194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:00.410632 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:00.390285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:56961 (local address 127.0.0.1:35221)
0807 04:15:00.390524 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:00.390531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:00.390962 (+ 431us) server_negotiation.cc:407] Connection header received
0807 04:15:00.391124 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:00.391128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:00.391200 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:00.391294 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:00.393440 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:00.394307 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:00.396209 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:00.397065 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:00.397240 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:00.401537 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:00.401547 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:00.401554 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:00.401606 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:00.405984 (+ 4378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:00.407182 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:00.407190 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:00.407195 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:00.407367 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:00.408048 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:00.408053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:00.408056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:00.408547 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:00.410092 (+ 1545us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:00.410246 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 04:15:00.410375 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 04:15:00.413709 9201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:00.419094 9201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:00.431833 9201 catalog_manager.cc:1347] Generated new cluster ID: 48c074cbab544d53b977ffb02fa0e5e4
I20260807 04:15:00.431933 9201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:00.449110 9201 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:00.451138 9201 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:00.475778 9201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d496445c46c471883788c1e64afb552: Generated new TSK 0
I20260807 04:15:00.476776 9201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:00.490937 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9111
I20260807 04:15:00.492734 9176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:00.585824 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9111
W20260807 04:15:00.606206 9096 connection.cc:535] client connection to 127.0.0.1:35221 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35221): Connection reset by peer (error 104)
2026-08-07T04:15:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2155 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:00 dist-test-slave-8n49 krb5kdc[9215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:00 dist-test-slave-8n49 krb5kdc[9215](info): set up 2 sockets
Aug 07 04:15:00 dist-test-slave-8n49 krb5kdc[9215](info): commencing operation
krb5kdc: starting...
W20260807 04:15:01.750430 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:01 dist-test-slave-8n49 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:15:02.060999 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:02.401952 9228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:02.402307 9228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:02.402486 9228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:02.409755 9228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:02.409920 9228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:02.410034 9228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:02.410146 9228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:02.410235 9228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:02.420126 9228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:02.421494 9228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:02.422809 9228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:02.434839 9234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:02.434800 9235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:02.434815 9237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:02.436084 9228 server_base.cc:1034] running on GCE node
I20260807 04:15:02.437022 9228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:02.438535 9228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:02.439807 9228 hybrid_clock.cc:648] HybridClock initialized: now 1786076102439747 us; error 47 us; skew 500 ppm
I20260807 04:15:02.440244 9228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:02 dist-test-slave-8n49 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076102, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:02.448904 9228 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:15:02.450701 9228 webserver.cc:460] Webserver started at http://127.0.0.1:33591/ using document root <none> and password file <none>
I20260807 04:15:02.451454 9228 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:02.451611 9228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:02.451938 9228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:02.455183 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "8917a49eab174b55be052c5c4f3d70b4"
format_stamp: "Formatted at 2026-08-07 04:15:02 on dist-test-slave-8n49"
server_key: "765bba405b31c9c51128c6bf95ae467e"
server_key_iv: "bf271a1f61584f4f4f9cb8ea37fe3e6f"
server_key_version: "encryptionkey@0"
I20260807 04:15:02.456028 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "8917a49eab174b55be052c5c4f3d70b4"
format_stamp: "Formatted at 2026-08-07 04:15:02 on dist-test-slave-8n49"
server_key: "765bba405b31c9c51128c6bf95ae467e"
server_key_iv: "bf271a1f61584f4f4f9cb8ea37fe3e6f"
server_key_version: "encryptionkey@0"
I20260807 04:15:02.461583 9228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 04:15:02.465194 9244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:02.466740 9228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:15:02.466953 9228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "8917a49eab174b55be052c5c4f3d70b4"
format_stamp: "Formatted at 2026-08-07 04:15:02 on dist-test-slave-8n49"
server_key: "765bba405b31c9c51128c6bf95ae467e"
server_key_iv: "bf271a1f61584f4f4f9cb8ea37fe3e6f"
server_key_version: "encryptionkey@0"
I20260807 04:15:02.467180 9228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:02.507550 9228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:02.512717 9228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:02.513077 9228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:02.533960 9228 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41033
I20260807 04:15:02.533979 9298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41033 every 8 connection(s)
I20260807 04:15:02.535511 9228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:02.539305 9299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:02.542186 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9228
I20260807 04:15:02.542382 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:02.542936 19204 external_mini_cluster.cc:1428] Setting key 5c71906a711be3ef3b02ec95bf846c54
I20260807 04:15:02.551847 9299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4: Bootstrap starting.
I20260807 04:15:02.555763 9299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:02.557008 9299 log.cc:824] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:02.557893 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:02.545020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:60697 (local address 127.0.0.1:41033)
0807 04:15:02.545930 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:02.545942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:02.545986 (+ 44us) server_negotiation.cc:407] Connection header received
0807 04:15:02.547828 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:02.547851 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:02.548306 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:02.548810 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:02.549391 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:02.550891 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:02.552843 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:02.553627 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:02.553780 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:02.555163 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:02.555171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:02.555185 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:02.555742 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:02.555822 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:02.555915 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:02.556109 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:02.556418 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 04:15:02.556654 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":442,"threads_started":1}
I20260807 04:15:02.560127 9299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4: No bootstrap required, opened a new log
I20260807 04:15:02.561926 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:02.568774 9299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917a49eab174b55be052c5c4f3d70b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260807 04:15:02.569262 9299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:02.569424 9299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8917a49eab174b55be052c5c4f3d70b4, State: Initialized, Role: FOLLOWER
I20260807 04:15:02.570001 9299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917a49eab174b55be052c5c4f3d70b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260807 04:15:02.570261 9299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:02.570421 9299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:02.570597 9299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:02.573824 9299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917a49eab174b55be052c5c4f3d70b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260807 04:15:02.573946 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:02.566417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:33541 (local address 127.0.0.1:41033)
0807 04:15:02.566613 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:02.566624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:02.567142 (+ 518us) server_negotiation.cc:407] Connection header received
0807 04:15:02.567297 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:02.567306 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:02.567394 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:02.567515 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:02.568000 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:02.569151 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:02.570929 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:02.571650 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:02.571794 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:02.573044 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:02.573050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:02.573056 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:02.573078 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:02.573148 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:02.573168 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:02.573277 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:02.573520 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 04:15:02.573582 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:15:02.574484 9299 leader_election.cc:302] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8917a49eab174b55be052c5c4f3d70b4; no voters:
I20260807 04:15:02.575146 9299 leader_election.cc:288] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:02.575310 9311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:02.576164 9311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [term 1 LEADER]: Becoming Leader. State: Replica: 8917a49eab174b55be052c5c4f3d70b4, State: Running, Role: LEADER
I20260807 04:15:02.576849 9311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917a49eab174b55be052c5c4f3d70b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260807 04:15:02.577517 9299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:02.579710 9313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8917a49eab174b55be052c5c4f3d70b4. Latest consensus state: current_term: 1 leader_uuid: "8917a49eab174b55be052c5c4f3d70b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917a49eab174b55be052c5c4f3d70b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } } }
I20260807 04:15:02.580107 9313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:02.581739 9312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8917a49eab174b55be052c5c4f3d70b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8917a49eab174b55be052c5c4f3d70b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } } }
I20260807 04:15:02.582015 9316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:02.582135 9312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:02.587020 9316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:02.595851 9316 catalog_manager.cc:1347] Generated new cluster ID: 6c6aadfa14e24569b6f962d67703812c
I20260807 04:15:02.595963 9316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:02.611531 9316 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:02.613372 9316 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:02.631177 9316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8917a49eab174b55be052c5c4f3d70b4: Generated new TSK 0
I20260807 04:15:02.632254 9316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:02.642310 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9228
W20260807 04:15:02.643605 9294 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.25:60697
W20260807 04:15:02.644347 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9228: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:02.644517 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9228
W20260807 04:15:02.663996 9212 connection.cc:535] client connection to 127.0.0.1:41033 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41033): Connection reset by peer (error 104)
2026-08-07T04:15:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:02 dist-test-slave-8n49 krb5kdc[9332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:02 dist-test-slave-8n49 krb5kdc[9332](info): set up 2 sockets
Aug 07 04:15:02 dist-test-slave-8n49 krb5kdc[9332](info): commencing operation
krb5kdc: starting...
W20260807 04:15:03.802379 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:03 dist-test-slave-8n49 krb5kdc[9332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:15:04.103556 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:04.460613 9346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:04.460937 9346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:04.461033 9346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:04.468432 9346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:04.468611 9346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:04.468686 9346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:04.468752 9346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:04.468797 9346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:04.479215 9346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:04.480512 9346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:04.481989 9346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:04.494381 9351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:04.494302 9352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:04.494351 9354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:04.495146 9346 server_base.cc:1034] running on GCE node
I20260807 04:15:04.495975 9346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:04.497443 9346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:04.498680 9346 hybrid_clock.cc:648] HybridClock initialized: now 1786076104498650 us; error 53 us; skew 500 ppm
I20260807 04:15:04.499043 9346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:04 dist-test-slave-8n49 krb5kdc[9332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:04.502350 9346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:15:04.503883 9346 webserver.cc:460] Webserver started at http://127.0.0.1:41501/ using document root <none> and password file <none>
I20260807 04:15:04.504539 9346 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:04.504720 9346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:04.504992 9346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:04.507957 9346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "ad974e39378d4f3f9547af93c6a0e921"
format_stamp: "Formatted at 2026-08-07 04:15:04 on dist-test-slave-8n49"
server_key: "60b277084d6e5ebf33312d619436ef34"
server_key_iv: "8a7b81c3625565c519888b0fdbb0b8de"
server_key_version: "encryptionkey@0"
I20260807 04:15:04.508786 9346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "ad974e39378d4f3f9547af93c6a0e921"
format_stamp: "Formatted at 2026-08-07 04:15:04 on dist-test-slave-8n49"
server_key: "60b277084d6e5ebf33312d619436ef34"
server_key_iv: "8a7b81c3625565c519888b0fdbb0b8de"
server_key_version: "encryptionkey@0"
I20260807 04:15:04.514005 9346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:15:04.517560 9361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:04.518910 9346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:15:04.519089 9346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "ad974e39378d4f3f9547af93c6a0e921"
format_stamp: "Formatted at 2026-08-07 04:15:04 on dist-test-slave-8n49"
server_key: "60b277084d6e5ebf33312d619436ef34"
server_key_iv: "8a7b81c3625565c519888b0fdbb0b8de"
server_key_version: "encryptionkey@0"
I20260807 04:15:04.519290 9346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:04.525162 9346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:04.529325 9346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:04.529645 9346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:04.548727 9346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44809
I20260807 04:15:04.548817 9415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44809 every 8 connection(s)
I20260807 04:15:04.550200 9346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:04.553169 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9346
I20260807 04:15:04.553363 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:04.553825 19204 external_mini_cluster.cc:1428] Setting key 4a985d2267447495191b074bbe1cc51e
I20260807 04:15:04.554211 9416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:04.563125 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:04.555747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:44101 (local address 127.0.0.1:44809)
0807 04:15:04.556591 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:04.556606 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:04.556657 (+ 51us) server_negotiation.cc:407] Connection header received
0807 04:15:04.559104 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:04.559135 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:04.559607 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:04.560084 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:04.561151 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:04.561159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:04.561175 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:04.561193 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:04.561268 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:04.561294 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:04.561550 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:04.561684 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 04:15:04.561904 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":333,"threads_started":1}
I20260807 04:15:04.567106 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:04.569149 9416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921: Bootstrap starting.
I20260807 04:15:04.572715 9416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:04.573880 9416 log.cc:824] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:04.574697 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:04.571875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:60523 (local address 127.0.0.1:44809)
0807 04:15:04.572058 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:04.572063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:04.572470 (+ 407us) server_negotiation.cc:407] Connection header received
0807 04:15:04.572629 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:04.572634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:04.572706 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:04.572823 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:04.573800 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:04.573806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:04.573813 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:04.573835 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:04.573890 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:04.573914 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:04.574035 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:04.574208 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 04:15:04.574268 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:15:04.577154 9416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921: No bootstrap required, opened a new log
I20260807 04:15:04.585011 9416 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad974e39378d4f3f9547af93c6a0e921" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260807 04:15:04.585376 9416 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:04.585453 9416 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad974e39378d4f3f9547af93c6a0e921, State: Initialized, Role: FOLLOWER
I20260807 04:15:04.585986 9416 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad974e39378d4f3f9547af93c6a0e921" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260807 04:15:04.586184 9416 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:04.586280 9416 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:04.586397 9416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:04.588811 9416 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad974e39378d4f3f9547af93c6a0e921" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260807 04:15:04.589361 9416 leader_election.cc:302] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad974e39378d4f3f9547af93c6a0e921; no voters:
I20260807 04:15:04.590016 9416 leader_election.cc:288] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:04.590164 9426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:04.590938 9426 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [term 1 LEADER]: Becoming Leader. State: Replica: ad974e39378d4f3f9547af93c6a0e921, State: Running, Role: LEADER
I20260807 04:15:04.591514 9426 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad974e39378d4f3f9547af93c6a0e921" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260807 04:15:04.592134 9416 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:04.594558 9427 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad974e39378d4f3f9547af93c6a0e921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad974e39378d4f3f9547af93c6a0e921" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } } }
I20260807 04:15:04.594610 9428 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad974e39378d4f3f9547af93c6a0e921. Latest consensus state: current_term: 1 leader_uuid: "ad974e39378d4f3f9547af93c6a0e921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad974e39378d4f3f9547af93c6a0e921" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } } }
I20260807 04:15:04.595055 9427 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:04.595117 9428 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:04.596108 9435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:04.602509 9435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:04.614464 9435 catalog_manager.cc:1347] Generated new cluster ID: 9ece5b53622e45a68e83b91584781817
I20260807 04:15:04.614609 9435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:04.643231 9435 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:04.643712 9435 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:04.652266 9435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad974e39378d4f3f9547af93c6a0e921: Generated new TSK 0
I20260807 04:15:04.653232 9435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:04.659986 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9346
W20260807 04:15:04.661166 9411 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.25:44101
W20260807 04:15:04.661878 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9346: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:04.662035 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9346
2026-08-07T04:15:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2018 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:04 dist-test-slave-8n49 krb5kdc[9448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:04 dist-test-slave-8n49 krb5kdc[9448](info): set up 2 sockets
Aug 07 04:15:04 dist-test-slave-8n49 krb5kdc[9448](info): commencing operation
krb5kdc: starting...
W20260807 04:15:05.825354 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:06 dist-test-slave-8n49 krb5kdc[9448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:15:06.143493 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:06.489341 9461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:06.489642 9461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:06.489833 9461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:06.497308 9461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:06.497433 9461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:06.497567 9461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:06.497670 9461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:06.497756 9461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:06.508045 9461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:06.509467 9461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:06.510830 9461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:06.523010 9466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:06.523059 9467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:06.523108 9469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:06.524144 9461 server_base.cc:1034] running on GCE node
I20260807 04:15:06.524998 9461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:06.526428 9461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:06.527673 9461 hybrid_clock.cc:648] HybridClock initialized: now 1786076106527634 us; error 49 us; skew 500 ppm
I20260807 04:15:06.528021 9461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:06 dist-test-slave-8n49 krb5kdc[9448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:06.531250 9461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:15:06.532869 9461 webserver.cc:460] Webserver started at http://127.0.0.1:46841/ using document root <none> and password file <none>
I20260807 04:15:06.533586 9461 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:06.533737 9461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:06.534056 9461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:06.537220 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "5a0babf8f7844366a110da7cbe366090"
format_stamp: "Formatted at 2026-08-07 04:15:06 on dist-test-slave-8n49"
server_key: "8edb07e220faf636bec93ba285fa54b5"
server_key_iv: "6af661e04235a503ceec1e9f08e17a7a"
server_key_version: "encryptionkey@0"
I20260807 04:15:06.538017 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "5a0babf8f7844366a110da7cbe366090"
format_stamp: "Formatted at 2026-08-07 04:15:06 on dist-test-slave-8n49"
server_key: "8edb07e220faf636bec93ba285fa54b5"
server_key_iv: "6af661e04235a503ceec1e9f08e17a7a"
server_key_version: "encryptionkey@0"
I20260807 04:15:06.543398 9461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:15:06.546898 9477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:06.548177 9461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:15:06.548375 9461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "5a0babf8f7844366a110da7cbe366090"
format_stamp: "Formatted at 2026-08-07 04:15:06 on dist-test-slave-8n49"
server_key: "8edb07e220faf636bec93ba285fa54b5"
server_key_iv: "6af661e04235a503ceec1e9f08e17a7a"
server_key_version: "encryptionkey@0"
I20260807 04:15:06.548657 9461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:06.573201 9461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:06.577951 9461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:06.578267 9461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:06.597635 9461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34087
I20260807 04:15:06.597671 9532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34087 every 8 connection(s)
I20260807 04:15:06.599063 9461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:06.602942 9533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:06.604722 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9461
I20260807 04:15:06.604944 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:06.605417 19204 external_mini_cluster.cc:1428] Setting key a4f12dc80ad0dc1c94e31188afd07e9f
I20260807 04:15:06.617231 9533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090: Bootstrap starting.
Aug 07 04:15:06 dist-test-slave-8n49 krb5kdc[9448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:15:06.621191 9533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:06.622336 9533 log.cc:824] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:06.625501 9533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090: No bootstrap required, opened a new log
I20260807 04:15:06.629379 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:06.607309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:58633 (local address 127.0.0.1:34087)
0807 04:15:06.608165 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:06.608176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:06.608219 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:15:06.609883 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:06.609912 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:06.610313 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:06.610659 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:06.612507 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:06.613906 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:06.615704 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:06.616352 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:06.616527 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:06.621638 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:06.621646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:06.621663 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:06.621706 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:06.624958 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:06.625858 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:06.625865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:06.625871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:06.626054 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:06.626554 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:06.626558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:06.626560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:06.627070 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:06.627464 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:06.627961 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 04:15:06.628299 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":395,"threads_started":1}
I20260807 04:15:06.633762 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:06.634567 9533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0babf8f7844366a110da7cbe366090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260807 04:15:06.634959 9533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:06.635080 9533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0babf8f7844366a110da7cbe366090, State: Initialized, Role: FOLLOWER
I20260807 04:15:06.635735 9533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0babf8f7844366a110da7cbe366090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260807 04:15:06.635988 9533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:06.636140 9533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:06.636332 9533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:06.638830 9533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0babf8f7844366a110da7cbe366090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260807 04:15:06.639452 9533 leader_election.cc:302] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a0babf8f7844366a110da7cbe366090; no voters:
I20260807 04:15:06.640166 9533 leader_election.cc:288] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:06.640307 9544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:06.642688 9533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:06.642735 9544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [term 1 LEADER]: Becoming Leader. State: Replica: 5a0babf8f7844366a110da7cbe366090, State: Running, Role: LEADER
I20260807 04:15:06.643350 9544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0babf8f7844366a110da7cbe366090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260807 04:15:06.650065 9546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a0babf8f7844366a110da7cbe366090. Latest consensus state: current_term: 1 leader_uuid: "5a0babf8f7844366a110da7cbe366090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0babf8f7844366a110da7cbe366090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } } }
I20260807 04:15:06.650630 9546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:06.651528 9545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a0babf8f7844366a110da7cbe366090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0babf8f7844366a110da7cbe366090" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } } }
I20260807 04:15:06.652047 9545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:06.657732 9552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:06.662549 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:06.638854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:49153 (local address 127.0.0.1:34087)
0807 04:15:06.639071 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:06.639078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:06.639475 (+ 397us) server_negotiation.cc:407] Connection header received
0807 04:15:06.639613 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:06.639621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:06.639707 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:06.639826 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:06.642192 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:06.643757 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:06.646261 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:06.647202 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:06.647399 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:06.651986 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:06.651996 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:06.652004 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:06.652058 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:06.657283 (+ 5225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:06.658507 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:06.658513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:06.658516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:06.658685 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:06.659407 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:06.659415 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:06.659421 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:06.660017 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:06.661710 (+ 1693us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:06.661920 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 04:15:06.662133 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:15:06.664417 9552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:06.674623 9552 catalog_manager.cc:1347] Generated new cluster ID: 6606abe3671248eca0315c58d0191d6c
I20260807 04:15:06.674752 9552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:06.687482 9552 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:06.689913 9552 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:06.708690 9552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a0babf8f7844366a110da7cbe366090: Generated new TSK 0
I20260807 04:15:06.709872 9552 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:06.722882 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9461
I20260807 04:15:06.724107 9527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:06.819233 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9461
W20260807 04:15:06.839673 9445 connection.cc:535] client connection to 127.0.0.1:34087 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34087): Connection reset by peer (error 104)
2026-08-07T04:15:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2156 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:06 dist-test-slave-8n49 krb5kdc[9567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:06 dist-test-slave-8n49 krb5kdc[9567](info): set up 2 sockets
Aug 07 04:15:06 dist-test-slave-8n49 krb5kdc[9567](info): commencing operation
krb5kdc: starting...
W20260807 04:15:07.969975 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:08 dist-test-slave-8n49 krb5kdc[9567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:15:08.287209 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:08.646746 9580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:08.647055 9580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:08.647264 9580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:08.654696 9580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:08.654819 9580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:08.654999 9580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:08.655104 9580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:08.655170 9580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:08.665359 9580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:08.666723 9580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:08.668064 9580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:08.680123 9585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:08.680097 9588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:08.680308 9586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:08.681484 9580 server_base.cc:1034] running on GCE node
I20260807 04:15:08.682265 9580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:08.683701 9580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:08.684901 9580 hybrid_clock.cc:648] HybridClock initialized: now 1786076108684855 us; error 34 us; skew 500 ppm
I20260807 04:15:08.685274 9580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:08 dist-test-slave-8n49 krb5kdc[9567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:08.688441 9580 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:15:08.690037 9580 webserver.cc:460] Webserver started at http://127.0.0.1:43029/ using document root <none> and password file <none>
I20260807 04:15:08.690729 9580 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:08.690878 9580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:08.691192 9580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:08.694303 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "fbb60c3335da4457820f5a8dd3884d5c"
format_stamp: "Formatted at 2026-08-07 04:15:08 on dist-test-slave-8n49"
server_key: "8465413f81e0c49aa8a29af370ae6106"
server_key_iv: "68e58822eeee187fbc4c1a43c8f3f982"
server_key_version: "encryptionkey@0"
I20260807 04:15:08.695089 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "fbb60c3335da4457820f5a8dd3884d5c"
format_stamp: "Formatted at 2026-08-07 04:15:08 on dist-test-slave-8n49"
server_key: "8465413f81e0c49aa8a29af370ae6106"
server_key_iv: "68e58822eeee187fbc4c1a43c8f3f982"
server_key_version: "encryptionkey@0"
I20260807 04:15:08.700254 9580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 04:15:08.703794 9595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:08.705188 9580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:15:08.705381 9580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "fbb60c3335da4457820f5a8dd3884d5c"
format_stamp: "Formatted at 2026-08-07 04:15:08 on dist-test-slave-8n49"
server_key: "8465413f81e0c49aa8a29af370ae6106"
server_key_iv: "68e58822eeee187fbc4c1a43c8f3f982"
server_key_version: "encryptionkey@0"
I20260807 04:15:08.705586 9580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:08.729727 9580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:08.734004 9580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:08.734306 9580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:08.753765 9580 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43521
I20260807 04:15:08.753796 9651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43521 every 8 connection(s)
I20260807 04:15:08.755221 9580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:08.757781 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9580
I20260807 04:15:08.757984 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:08.758566 19204 external_mini_cluster.cc:1428] Setting key ae4f6b15abcaeeb08288b0d95a844b2c
I20260807 04:15:08.759733 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:08.772404 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:08.760363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:57157 (local address 127.0.0.1:43521)
0807 04:15:08.761248 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:08.761261 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:08.761302 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:15:08.762948 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:08.762969 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:08.763390 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:08.763732 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:08.764360 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:08.765784 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:08.767688 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:08.768356 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:08.768490 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:08.769878 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:08.769886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:08.769906 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:08.770563 (+ 657us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:08.770641 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:08.770669 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:08.770844 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:08.771201 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 04:15:08.771403 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":430,"threads_started":1}
I20260807 04:15:08.773793 9653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c: Bootstrap starting.
I20260807 04:15:08.776292 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:08.777221 9653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:08.778210 9653 log.cc:824] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:08.781592 9653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c: No bootstrap required, opened a new log
I20260807 04:15:08.787171 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:08.780971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:54061 (local address 127.0.0.1:43521)
0807 04:15:08.781150 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:08.781158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:08.781564 (+ 406us) server_negotiation.cc:407] Connection header received
0807 04:15:08.781781 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:08.781790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:08.781849 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:08.781961 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:08.782474 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:08.783245 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:08.784593 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:08.785069 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:08.785204 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:08.786397 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:08.786401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:08.786405 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:08.786423 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:08.786477 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:08.786496 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:08.786626 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:08.786836 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 04:15:08.786894 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 04:15:08.789491 9653 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb60c3335da4457820f5a8dd3884d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260807 04:15:08.789898 9653 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:08.790009 9653 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb60c3335da4457820f5a8dd3884d5c, State: Initialized, Role: FOLLOWER
I20260807 04:15:08.790553 9653 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb60c3335da4457820f5a8dd3884d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260807 04:15:08.790822 9653 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:08.790975 9653 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:08.791152 9653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:08.793505 9653 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb60c3335da4457820f5a8dd3884d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260807 04:15:08.794082 9653 leader_election.cc:302] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbb60c3335da4457820f5a8dd3884d5c; no voters:
I20260807 04:15:08.794800 9653 leader_election.cc:288] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:08.794907 9663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:08.795797 9663 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [term 1 LEADER]: Becoming Leader. State: Replica: fbb60c3335da4457820f5a8dd3884d5c, State: Running, Role: LEADER
I20260807 04:15:08.796345 9663 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb60c3335da4457820f5a8dd3884d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260807 04:15:08.797029 9653 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:08.799176 9664 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbb60c3335da4457820f5a8dd3884d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb60c3335da4457820f5a8dd3884d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260807 04:15:08.799400 9665 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbb60c3335da4457820f5a8dd3884d5c. Latest consensus state: current_term: 1 leader_uuid: "fbb60c3335da4457820f5a8dd3884d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb60c3335da4457820f5a8dd3884d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260807 04:15:08.799710 9664 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:08.799738 9665 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:08.800678 9670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:08.806485 9670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:08.815706 9670 catalog_manager.cc:1347] Generated new cluster ID: ac4737e0e2bd49178223126335d74062
I20260807 04:15:08.815829 9670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:08.846203 9670 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:08.847528 9670 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:08.859222 9670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbb60c3335da4457820f5a8dd3884d5c: Generated new TSK 0
I20260807 04:15:08.860117 9670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:08.872968 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9580
W20260807 04:15:08.874123 9646 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.25:57157
W20260807 04:15:08.874926 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9580: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:08.875093 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9580
W20260807 04:15:08.895398 9564 connection.cc:535] client connection to 127.0.0.1:43521 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43521): Connection reset by peer (error 104)
2026-08-07T04:15:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2055 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:08 dist-test-slave-8n49 krb5kdc[9686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:08 dist-test-slave-8n49 krb5kdc[9686](info): set up 2 sockets
Aug 07 04:15:08 dist-test-slave-8n49 krb5kdc[9686](info): commencing operation
krb5kdc: starting...
W20260807 04:15:10.025990 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:10 dist-test-slave-8n49 krb5kdc[9686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:15:10.328169 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33563
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:10.670221 9702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:10.670594 9702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:10.670758 9702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:10.678079 9702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:10.678231 9702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:10.678339 9702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:10.678464 9702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:10.678565 9702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:10.688419 9702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33563
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:10.689742 9702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:10.691035 9702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:10.702929 9707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:10.702910 9708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:10.703106 9710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:10.703559 9702 server_base.cc:1034] running on GCE node
I20260807 04:15:10.704423 9702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:10.705868 9702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:10.707075 9702 hybrid_clock.cc:648] HybridClock initialized: now 1786076110707015 us; error 51 us; skew 500 ppm
I20260807 04:15:10.707460 9702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:10 dist-test-slave-8n49 krb5kdc[9686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:10.710697 9702 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:15:10.712466 9702 webserver.cc:460] Webserver started at http://127.0.0.1:40867/ using document root <none> and password file <none>
I20260807 04:15:10.713299 9702 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:10.713406 9702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:10.713656 9702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:10.716634 9702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "c4e22fb5189747afb285b8e4919182b4"
format_stamp: "Formatted at 2026-08-07 04:15:10 on dist-test-slave-8n49"
server_key: "70ac6829946b01e6a74f071a04b67f80"
server_key_iv: "6ab0e4defa118e485b2b99e26c8ae8bb"
server_key_version: "encryptionkey@0"
I20260807 04:15:10.717370 9702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "c4e22fb5189747afb285b8e4919182b4"
format_stamp: "Formatted at 2026-08-07 04:15:10 on dist-test-slave-8n49"
server_key: "70ac6829946b01e6a74f071a04b67f80"
server_key_iv: "6ab0e4defa118e485b2b99e26c8ae8bb"
server_key_version: "encryptionkey@0"
I20260807 04:15:10.722466 9702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:15:10.725858 9717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:10.727195 9702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:15:10.727380 9702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "c4e22fb5189747afb285b8e4919182b4"
format_stamp: "Formatted at 2026-08-07 04:15:10 on dist-test-slave-8n49"
server_key: "70ac6829946b01e6a74f071a04b67f80"
server_key_iv: "6ab0e4defa118e485b2b99e26c8ae8bb"
server_key_version: "encryptionkey@0"
I20260807 04:15:10.727594 9702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:10.733155 9702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:10.737111 9702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:10.737411 9702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:10.755744 9702 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33563
I20260807 04:15:10.755787 9773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33563 every 8 connection(s)
I20260807 04:15:10.757185 9702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:10.757563 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9702
I20260807 04:15:10.757776 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:10.758385 19204 external_mini_cluster.cc:1428] Setting key 5a864203be412bcc8d652d302e9c55aa
I20260807 04:15:10.762058 9774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:10.766481 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:10.760241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:33391 (local address 127.0.0.1:33563)
0807 04:15:10.761120 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:10.761136 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:10.761189 (+ 53us) server_negotiation.cc:407] Connection header received
0807 04:15:10.763178 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:10.763200 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:10.763512 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:10.763826 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:10.764930 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:10.764936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:10.764947 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:10.764964 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:10.765058 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:10.765081 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:10.765248 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:10.765385 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 04:15:10.765585 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":402,"threads_started":1}
I20260807 04:15:10.770463 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:10.775635 9774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4: Bootstrap starting.
I20260807 04:15:10.778453 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:10.775899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.25:59923 (local address 127.0.0.1:33563)
0807 04:15:10.776092 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:10.776098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:10.776501 (+ 403us) server_negotiation.cc:407] Connection header received
0807 04:15:10.776698 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:10.776704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:10.776769 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:10.776881 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:10.777809 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:10.777814 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:10.777818 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:10.777836 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:10.777883 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:10.777902 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:10.778017 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:10.778105 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 04:15:10.778165 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:15:10.779457 9774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:10.780782 9774 log.cc:824] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:10.783548 9774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4: No bootstrap required, opened a new log
I20260807 04:15:10.791109 9774 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e22fb5189747afb285b8e4919182b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33563 } }
I20260807 04:15:10.791502 9774 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:10.791608 9774 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e22fb5189747afb285b8e4919182b4, State: Initialized, Role: FOLLOWER
I20260807 04:15:10.792155 9774 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e22fb5189747afb285b8e4919182b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33563 } }
I20260807 04:15:10.792414 9774 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:10.792608 9774 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:10.792806 9774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:10.795171 9774 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e22fb5189747afb285b8e4919182b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33563 } }
I20260807 04:15:10.795676 9774 leader_election.cc:302] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e22fb5189747afb285b8e4919182b4; no voters:
I20260807 04:15:10.796293 9774 leader_election.cc:288] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:10.796460 9785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:10.797174 9785 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [term 1 LEADER]: Becoming Leader. State: Replica: c4e22fb5189747afb285b8e4919182b4, State: Running, Role: LEADER
I20260807 04:15:10.797698 9785 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e22fb5189747afb285b8e4919182b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33563 } }
I20260807 04:15:10.798353 9774 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:10.800412 9787 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e22fb5189747afb285b8e4919182b4. Latest consensus state: current_term: 1 leader_uuid: "c4e22fb5189747afb285b8e4919182b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e22fb5189747afb285b8e4919182b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33563 } } }
I20260807 04:15:10.800666 9786 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e22fb5189747afb285b8e4919182b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e22fb5189747afb285b8e4919182b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33563 } } }
I20260807 04:15:10.800899 9787 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:10.800945 9786 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:10.802100 9793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:10.807248 9793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:10.815223 9793 catalog_manager.cc:1347] Generated new cluster ID: fe314e196f07468a9938921c82e71123
I20260807 04:15:10.815335 9793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:10.843286 9793 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:10.843828 9793 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:10.865037 9793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e22fb5189747afb285b8e4919182b4: Generated new TSK 0
I20260807 04:15:10.866021 9793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:10.875732 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9702
W20260807 04:15:10.877149 9769 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.25:33391
W20260807 04:15:10.877861 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9702: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:10.878031 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9702
2026-08-07T04:15:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2003 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:10 dist-test-slave-8n49 krb5kdc[9806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:10 dist-test-slave-8n49 krb5kdc[9806](info): set up 2 sockets
Aug 07 04:15:10 dist-test-slave-8n49 krb5kdc[9806](info): commencing operation
krb5kdc: starting...
W20260807 04:15:12.038739 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:12 dist-test-slave-8n49 krb5kdc[9806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:12.338440 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34307
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:12.680965 9820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:12.681264 9820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:12.681454 9820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:12.688952 9820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:12.689069 9820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:12.689154 9820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:12.689258 9820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:12.689339 9820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:12.699452 9820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:34307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:12.700873 9820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:12.702183 9820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:12.714016 9827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:12.713973 9829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:12.714077 9826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:12.714771 9820 server_base.cc:1034] running on GCE node
I20260807 04:15:12.715500 9820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:12.716921 9820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:12.718178 9820 hybrid_clock.cc:648] HybridClock initialized: now 1786076112718123 us; error 58 us; skew 500 ppm
I20260807 04:15:12.718611 9820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:12 dist-test-slave-8n49 krb5kdc[9806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076112, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:12.721885 9820 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:12.723371 9820 webserver.cc:460] Webserver started at http://127.18.193.62:42975/ using document root <none> and password file <none>
I20260807 04:15:12.724073 9820 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:12.724224 9820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:12.724593 9820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:12.727568 9820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "0864526fb4204e00b974b6801dac5e3a"
format_stamp: "Formatted at 2026-08-07 04:15:12 on dist-test-slave-8n49"
server_key: "4e639eac1939d5d668cba4a29fe69d06"
server_key_iv: "add5210957249d55767dc83e293d5fe6"
server_key_version: "encryptionkey@0"
I20260807 04:15:12.728345 9820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "0864526fb4204e00b974b6801dac5e3a"
format_stamp: "Formatted at 2026-08-07 04:15:12 on dist-test-slave-8n49"
server_key: "4e639eac1939d5d668cba4a29fe69d06"
server_key_iv: "add5210957249d55767dc83e293d5fe6"
server_key_version: "encryptionkey@0"
I20260807 04:15:12.733579 9820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 04:15:12.737052 9837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:12.738317 9820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:15:12.738505 9820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "0864526fb4204e00b974b6801dac5e3a"
format_stamp: "Formatted at 2026-08-07 04:15:12 on dist-test-slave-8n49"
server_key: "4e639eac1939d5d668cba4a29fe69d06"
server_key_iv: "add5210957249d55767dc83e293d5fe6"
server_key_version: "encryptionkey@0"
I20260807 04:15:12.738714 9820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:12.762012 9820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:12.766618 9820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:12.766921 9820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:12.785815 9820 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:34307
I20260807 04:15:12.785840 9891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:34307 every 8 connection(s)
I20260807 04:15:12.787227 9820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:12.787488 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9820
I20260807 04:15:12.787681 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:12.788182 19204 external_mini_cluster.cc:1428] Setting key 6449b4863313fffc42e18e88b5ccb72c
I20260807 04:15:12.792826 9892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:15:12 dist-test-slave-8n49 krb5kdc[9806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:15:12.803320 9892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a: Bootstrap starting.
I20260807 04:15:12.806561 9892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:12.807606 9892 log.cc:824] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:12.809520 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:12.789872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36923 (local address 127.18.193.62:34307)
0807 04:15:12.790617 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:12.790628 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:12.790668 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:15:12.792310 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:12.792331 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:12.792777 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:12.793190 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:12.794658 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:12.795993 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:12.797315 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:12.797992 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:12.798185 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:12.802327 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:12.802335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:12.802348 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:12.802381 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:12.805448 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:12.806177 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:12.806184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:12.806189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:12.806345 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:12.806855 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:12.806860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:12.806862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:12.807358 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:12.807717 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:12.808151 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 04:15:12.808505 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":305,"threads_started":1}
I20260807 04:15:12.810611 9892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a: No bootstrap required, opened a new log
I20260807 04:15:12.813515 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:12.821619 9892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0864526fb4204e00b974b6801dac5e3a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34307 } }
I20260807 04:15:12.822129 9892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:12.822256 9892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0864526fb4204e00b974b6801dac5e3a, State: Initialized, Role: FOLLOWER
I20260807 04:15:12.822865 9892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0864526fb4204e00b974b6801dac5e3a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34307 } }
I20260807 04:15:12.823154 9892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:12.823302 9892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:12.823477 9892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:12.826602 9892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0864526fb4204e00b974b6801dac5e3a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34307 } }
I20260807 04:15:12.827232 9892 leader_election.cc:302] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0864526fb4204e00b974b6801dac5e3a; no voters:
I20260807 04:15:12.827939 9892 leader_election.cc:288] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:12.828043 9904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:12.829663 9904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [term 1 LEADER]: Becoming Leader. State: Replica: 0864526fb4204e00b974b6801dac5e3a, State: Running, Role: LEADER
I20260807 04:15:12.830212 9904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0864526fb4204e00b974b6801dac5e3a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34307 } }
I20260807 04:15:12.830425 9892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:12.834362 9905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0864526fb4204e00b974b6801dac5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0864526fb4204e00b974b6801dac5e3a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34307 } } }
I20260807 04:15:12.834781 9905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:12.837942 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:12.818891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53441 (local address 127.18.193.62:34307)
0807 04:15:12.819131 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:12.819141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:12.819489 (+ 348us) server_negotiation.cc:407] Connection header received
0807 04:15:12.819670 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:12.819674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:12.819737 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:12.819813 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:12.821794 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:12.822788 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:12.824640 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:12.825125 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:12.825237 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:12.828275 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:12.828282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:12.828289 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:12.828343 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:12.832814 (+ 4471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:12.834017 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:12.834023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:12.834029 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:12.834263 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:12.834895 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:12.834900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:12.834904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:12.835360 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:12.836957 (+ 1597us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:12.837185 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 04:15:12.837419 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:15:12.839881 9906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0864526fb4204e00b974b6801dac5e3a. Latest consensus state: current_term: 1 leader_uuid: "0864526fb4204e00b974b6801dac5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0864526fb4204e00b974b6801dac5e3a" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 34307 } } }
I20260807 04:15:12.840196 9906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:12.845477 9912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:12.849728 9912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:12.858871 9912 catalog_manager.cc:1347] Generated new cluster ID: 80110b7444c546e987feab49c4630a4c
I20260807 04:15:12.858989 9912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:12.875475 9912 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:12.877533 9912 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:12.886128 9912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0864526fb4204e00b974b6801dac5e3a: Generated new TSK 0
I20260807 04:15:12.887267 9912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:12.899698 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9820
I20260807 04:15:12.901098 9887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:12.992774 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9820
W20260807 04:15:13.012902 9804 connection.cc:535] client connection to 127.18.193.62:34307 recv error: Network error: failed to read from TLS socket (remote: 127.18.193.62:34307): Connection reset by peer (error 104)
2026-08-07T04:15:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2113 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:13 dist-test-slave-8n49 krb5kdc[9926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:13 dist-test-slave-8n49 krb5kdc[9926](info): set up 2 sockets
Aug 07 04:15:13 dist-test-slave-8n49 krb5kdc[9926](info): commencing operation
krb5kdc: starting...
W20260807 04:15:14.154704 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:14 dist-test-slave-8n49 krb5kdc[9926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:14.463936 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:46409
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:46409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:14.803491 9939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:14.803823 9939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:14.803999 9939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:14.811221 9939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:14.811367 9939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:14.811486 9939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:14.811589 9939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:14.811677 9939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:14.821571 9939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:46409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:46409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:14.822863 9939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:14.824179 9939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:14.835964 9944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:14.835927 9945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:14.836000 9947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:14.836781 9939 server_base.cc:1034] running on GCE node
I20260807 04:15:14.837543 9939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:14.838925 9939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:14.840188 9939 hybrid_clock.cc:648] HybridClock initialized: now 1786076114840147 us; error 54 us; skew 500 ppm
I20260807 04:15:14.840657 9939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:14 dist-test-slave-8n49 krb5kdc[9926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076114, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:14.846910 9939 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:14.848636 9939 webserver.cc:460] Webserver started at http://127.18.193.62:41999/ using document root <none> and password file <none>
I20260807 04:15:14.849392 9939 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:14.849560 9939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:14.849900 9939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:14.853250 9939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "74ddf914a090451e95486754e8d40f73"
format_stamp: "Formatted at 2026-08-07 04:15:14 on dist-test-slave-8n49"
server_key: "a9288aed6f6c2e3b5d658c68d773a7b5"
server_key_iv: "291f008d2c093ae9c684988f32b55446"
server_key_version: "encryptionkey@0"
I20260807 04:15:14.854099 9939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "74ddf914a090451e95486754e8d40f73"
format_stamp: "Formatted at 2026-08-07 04:15:14 on dist-test-slave-8n49"
server_key: "a9288aed6f6c2e3b5d658c68d773a7b5"
server_key_iv: "291f008d2c093ae9c684988f32b55446"
server_key_version: "encryptionkey@0"
I20260807 04:15:14.859597 9939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:15:14.863082 9954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:14.864363 9939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:15:14.864639 9939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "74ddf914a090451e95486754e8d40f73"
format_stamp: "Formatted at 2026-08-07 04:15:14 on dist-test-slave-8n49"
server_key: "a9288aed6f6c2e3b5d658c68d773a7b5"
server_key_iv: "291f008d2c093ae9c684988f32b55446"
server_key_version: "encryptionkey@0"
I20260807 04:15:14.864841 9939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:14.882802 9939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:14.887216 9939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:14.887518 9939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:14.906797 9939 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:46409
I20260807 04:15:14.906823 10008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:46409 every 8 connection(s)
I20260807 04:15:14.908183 9939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:14.911849 10009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:14.912632 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 9939
I20260807 04:15:14.912835 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:14.913300 19204 external_mini_cluster.cc:1428] Setting key 8302a0c745460411774fa642fd598d9f
I20260807 04:15:14.925338 10009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73: Bootstrap starting.
I20260807 04:15:14.926851 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:14.914980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49485 (local address 127.18.193.62:46409)
0807 04:15:14.915785 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:14.915796 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:14.915838 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:15:14.917869 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:14.917893 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:14.918263 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:14.918597 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:14.919180 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:14.920431 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:14.922457 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:14.923152 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:14.923280 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:14.924652 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:14.924659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:14.924672 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:14.925192 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:14.925264 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:14.925287 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:14.925416 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:14.925743 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 04:15:14.925932 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":385,"threads_started":1}
I20260807 04:15:14.928645 10009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:14.929879 10009 log.cc:824] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:14.930553 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:14.932641 10009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73: No bootstrap required, opened a new log
I20260807 04:15:14.941984 10009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ddf914a090451e95486754e8d40f73" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46409 } }
I20260807 04:15:14.942492 10009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:14.942629 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:14.935619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58639 (local address 127.18.193.62:46409)
0807 04:15:14.935837 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:14.935844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:14.936230 (+ 386us) server_negotiation.cc:407] Connection header received
0807 04:15:14.936362 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:14.936366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:14.936422 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:14.936544 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:14.937215 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:14.938017 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:14.939939 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:14.940391 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:14.940525 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:14.941732 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:14.941736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:14.941739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:14.941755 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:14.941805 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:14.941820 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:14.941903 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:14.942136 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 04:15:14.942177 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:15:14.942631 10009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ddf914a090451e95486754e8d40f73, State: Initialized, Role: FOLLOWER
I20260807 04:15:14.943274 10009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ddf914a090451e95486754e8d40f73" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46409 } }
I20260807 04:15:14.943527 10009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:14.943686 10009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:14.943872 10009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:14.946208 10009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ddf914a090451e95486754e8d40f73" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46409 } }
I20260807 04:15:14.946817 10009 leader_election.cc:302] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74ddf914a090451e95486754e8d40f73; no voters:
I20260807 04:15:14.947388 10009 leader_election.cc:288] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:14.947528 10021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:14.948338 10021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [term 1 LEADER]: Becoming Leader. State: Replica: 74ddf914a090451e95486754e8d40f73, State: Running, Role: LEADER
I20260807 04:15:14.949041 10021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ddf914a090451e95486754e8d40f73" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46409 } }
I20260807 04:15:14.950093 10009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:14.952183 10023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ddf914a090451e95486754e8d40f73. Latest consensus state: current_term: 1 leader_uuid: "74ddf914a090451e95486754e8d40f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ddf914a090451e95486754e8d40f73" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46409 } } }
I20260807 04:15:14.952458 10022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74ddf914a090451e95486754e8d40f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ddf914a090451e95486754e8d40f73" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46409 } } }
I20260807 04:15:14.952734 10023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:14.952819 10022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:14.954360 10029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:14.959826 10029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:14.968115 10029 catalog_manager.cc:1347] Generated new cluster ID: c4ff46fdecdd48649c7bb755fb2e55be
I20260807 04:15:14.968262 10029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:14.976265 10029 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:14.978035 10029 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:14.987558 10029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74ddf914a090451e95486754e8d40f73: Generated new TSK 0
I20260807 04:15:14.988335 10029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:15.001075 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9939
W20260807 04:15:15.002556 10004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49485
W20260807 04:15:15.003297 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 9939: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:15.003460 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 9939
W20260807 04:15:15.023823 9924 connection.cc:535] client connection to 127.18.193.62:46409 recv error: Network error: failed to read from TLS socket (remote: 127.18.193.62:46409): Connection reset by peer (error 104)
2026-08-07T04:15:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2011 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:15 dist-test-slave-8n49 krb5kdc[10042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:15 dist-test-slave-8n49 krb5kdc[10042](info): set up 2 sockets
Aug 07 04:15:15 dist-test-slave-8n49 krb5kdc[10042](info): commencing operation
krb5kdc: starting...
W20260807 04:15:16.177754 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:16 dist-test-slave-8n49 krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:16.480312 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:40281
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:40281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:16.824913 10055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:16.825212 10055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:16.825389 10055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:16.832700 10055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:16.832806 10055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:16.832903 10055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:16.832996 10055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:16.833089 10055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:16.843057 10055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:40281
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:40281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:16.844537 10055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:16.845865 10055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:16.857604 10060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:16.857568 10063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:16.857664 10061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:16.859021 10055 server_base.cc:1034] running on GCE node
I20260807 04:15:16.859802 10055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:16.861258 10055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:16.862468 10055 hybrid_clock.cc:648] HybridClock initialized: now 1786076116862432 us; error 32 us; skew 500 ppm
I20260807 04:15:16.862823 10055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:16 dist-test-slave-8n49 krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076116, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:16.866549 10055 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:16.868120 10055 webserver.cc:460] Webserver started at http://127.18.193.62:37677/ using document root <none> and password file <none>
I20260807 04:15:16.868921 10055 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:16.869078 10055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:16.869387 10055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:16.872385 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "f607cae3aa364a6a8ac33f374f1cf42d"
format_stamp: "Formatted at 2026-08-07 04:15:16 on dist-test-slave-8n49"
server_key: "4baf9fc000bcbbf93dbf83f24b8e3815"
server_key_iv: "43d4df8800365596a8f17923451967dd"
server_key_version: "encryptionkey@0"
I20260807 04:15:16.873265 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "f607cae3aa364a6a8ac33f374f1cf42d"
format_stamp: "Formatted at 2026-08-07 04:15:16 on dist-test-slave-8n49"
server_key: "4baf9fc000bcbbf93dbf83f24b8e3815"
server_key_iv: "43d4df8800365596a8f17923451967dd"
server_key_version: "encryptionkey@0"
I20260807 04:15:16.878573 10055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:15:16.882171 10070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:16.883478 10055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:15:16.883678 10055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "f607cae3aa364a6a8ac33f374f1cf42d"
format_stamp: "Formatted at 2026-08-07 04:15:16 on dist-test-slave-8n49"
server_key: "4baf9fc000bcbbf93dbf83f24b8e3815"
server_key_iv: "43d4df8800365596a8f17923451967dd"
server_key_version: "encryptionkey@0"
I20260807 04:15:16.883940 10055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:16.890028 10055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:16.894505 10055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:16.894809 10055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:16.913771 10055 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:40281
I20260807 04:15:16.913812 10122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:40281 every 8 connection(s)
I20260807 04:15:16.915174 10055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:16.918816 10123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:16.920842 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10055
I20260807 04:15:16.921039 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:16.921620 19204 external_mini_cluster.cc:1428] Setting key 6185b5ea2a9691d31795a9d861a4123f
I20260807 04:15:16.929277 10127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:16.923396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48249 (local address 127.18.193.62:40281)
0807 04:15:16.924213 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:16.924224 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:16.924274 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:15:16.925880 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:16.925906 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:16.926251 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:16.926576 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:16.927674 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:16.927680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:16.927692 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:16.927706 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:16.927766 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:16.927785 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:16.927950 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:16.928094 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 04:15:16.928298 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":409,"threads_started":1}
I20260807 04:15:16.933032 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:16.933315 10123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d: Bootstrap starting.
I20260807 04:15:16.936586 10123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:16.937925 10123 log.cc:824] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:16.940198 10127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:16.937536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48273 (local address 127.18.193.62:40281)
0807 04:15:16.937694 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:16.937698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:16.938244 (+ 546us) server_negotiation.cc:407] Connection header received
0807 04:15:16.938454 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:16.938464 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:16.938512 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:16.938626 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:16.939560 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:16.939565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:16.939570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:16.939588 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:16.939635 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:16.939650 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:16.939771 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:16.939868 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 04:15:16.939936 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 04:15:16.941052 10123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d: No bootstrap required, opened a new log
I20260807 04:15:16.949065 10123 raft_consensus.cc:348] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40281 } }
I20260807 04:15:16.949430 10123 raft_consensus.cc:374] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:16.949503 10123 raft_consensus.cc:729] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f607cae3aa364a6a8ac33f374f1cf42d, State: Initialized, Role: FOLLOWER
I20260807 04:15:16.950002 10123 consensus_queue.cc:260] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40281 } }
I20260807 04:15:16.950268 10123 raft_consensus.cc:388] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:16.950378 10123 raft_consensus.cc:482] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:16.950510 10123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:16.953222 10123 raft_consensus.cc:504] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40281 } }
I20260807 04:15:16.953691 10123 leader_election.cc:302] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f607cae3aa364a6a8ac33f374f1cf42d; no voters:
I20260807 04:15:16.954217 10123 leader_election.cc:288] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:16.954353 10136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:16.955133 10136 raft_consensus.cc:686] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [term 1 LEADER]: Becoming Leader. State: Replica: f607cae3aa364a6a8ac33f374f1cf42d, State: Running, Role: LEADER
I20260807 04:15:16.955701 10136 consensus_queue.cc:237] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40281 } }
I20260807 04:15:16.956461 10123 sys_catalog.cc:564] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:16.958405 10137 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40281 } } }
I20260807 04:15:16.958834 10137 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:16.960428 10138 sys_catalog.cc:455] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f607cae3aa364a6a8ac33f374f1cf42d. Latest consensus state: current_term: 1 leader_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f607cae3aa364a6a8ac33f374f1cf42d" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 40281 } } }
I20260807 04:15:16.960863 10138 sys_catalog.cc:458] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:16.961005 10141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:16.966856 10141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:16.975507 10141 catalog_manager.cc:1347] Generated new cluster ID: 2aeb8c4661174ee0b13b60a21381fa25
I20260807 04:15:16.975632 10141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:16.987541 10141 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:16.988050 10141 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:17.002000 10141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f607cae3aa364a6a8ac33f374f1cf42d: Generated new TSK 0
I20260807 04:15:17.003052 10141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:17.017009 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10055
W20260807 04:15:17.018283 10118 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48249
W20260807 04:15:17.018965 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10055: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:17.019136 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10055
2026-08-07T04:15:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2015 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:17 dist-test-slave-8n49 krb5kdc[10160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:17 dist-test-slave-8n49 krb5kdc[10160](info): set up 2 sockets
Aug 07 04:15:17 dist-test-slave-8n49 krb5kdc[10160](info): commencing operation
krb5kdc: starting...
W20260807 04:15:18.194020 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:18 dist-test-slave-8n49 krb5kdc[10160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:18.517802 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44811
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:39177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:44811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:18.857658 10174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:18.858024 10174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:18.858202 10174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:18.865549 10174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:18.865737 10174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:18.865845 10174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:18.865957 10174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:18.866050 10174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:18.876148 10174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:44811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:18.877543 10174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:18.879014 10174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:18.890988 10179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:18.890918 10182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:18.891009 10180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:18.891871 10174 server_base.cc:1034] running on GCE node
I20260807 04:15:18.892753 10174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:18.894140 10174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:18.895375 10174 hybrid_clock.cc:648] HybridClock initialized: now 1786076118895335 us; error 62 us; skew 500 ppm
I20260807 04:15:18.895740 10174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:18 dist-test-slave-8n49 krb5kdc[10160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076118, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:18.899096 10174 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:18.900754 10174 webserver.cc:460] Webserver started at http://127.18.193.62:41003/ using document root <none> and password file <none>
I20260807 04:15:18.901485 10174 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:18.901607 10174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:18.901886 10174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:18.905511 10174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "7f0bd8994a054e588480ba5c6d065140"
format_stamp: "Formatted at 2026-08-07 04:15:18 on dist-test-slave-8n49"
server_key: "3a3e955146f6585592fe6e295e8383e2"
server_key_iv: "dab69de1b5b92f886aac5491aef6dd04"
server_key_version: "encryptionkey@0"
I20260807 04:15:18.906301 10174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "7f0bd8994a054e588480ba5c6d065140"
format_stamp: "Formatted at 2026-08-07 04:15:18 on dist-test-slave-8n49"
server_key: "3a3e955146f6585592fe6e295e8383e2"
server_key_iv: "dab69de1b5b92f886aac5491aef6dd04"
server_key_version: "encryptionkey@0"
I20260807 04:15:18.911872 10174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:15:18.916062 10189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:18.917693 10174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:15:18.917886 10174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "7f0bd8994a054e588480ba5c6d065140"
format_stamp: "Formatted at 2026-08-07 04:15:18 on dist-test-slave-8n49"
server_key: "3a3e955146f6585592fe6e295e8383e2"
server_key_iv: "dab69de1b5b92f886aac5491aef6dd04"
server_key_version: "encryptionkey@0"
I20260807 04:15:18.918107 10174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:18.932082 10174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:18.936674 10174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:18.936968 10174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:18.955567 10174 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:44811
I20260807 04:15:18.955601 10242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:44811 every 8 connection(s)
I20260807 04:15:18.957086 10174 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:18.960832 10243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:18.966496 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10174
I20260807 04:15:18.966727 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:18.967224 19204 external_mini_cluster.cc:1428] Setting key 1014bf7b6cdc727fb8d4440374a9a9c8
I20260807 04:15:18.972711 10243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140: Bootstrap starting.
I20260807 04:15:18.976044 10243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:18.977227 10243 log.cc:824] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:18.980360 10243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140: No bootstrap required, opened a new log
Aug 07 04:15:18 dist-test-slave-8n49 krb5kdc[10160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:15:18.989485 10243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0bd8994a054e588480ba5c6d065140" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44811 } }
I20260807 04:15:18.990012 10243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:18.990164 10243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0bd8994a054e588480ba5c6d065140, State: Initialized, Role: FOLLOWER
I20260807 04:15:18.990948 10243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0bd8994a054e588480ba5c6d065140" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44811 } }
I20260807 04:15:18.991219 10243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:18.991407 10243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:18.991613 10243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:18.992484 10246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:18.969163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33845 (local address 127.18.193.62:44811)
0807 04:15:18.970089 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:18.970101 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:18.970148 (+ 47us) server_negotiation.cc:407] Connection header received
0807 04:15:18.972026 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:18.972050 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:18.972500 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:18.973010 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:18.974638 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:18.976005 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:18.977538 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:18.978432 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:18.978686 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:18.983262 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:18.983298 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:18.983314 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:18.983364 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:18.986963 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:18.987766 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:18.987774 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:18.987781 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:18.987980 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:18.988433 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:18.988443 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:18.988445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:18.989378 (+ 933us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:18.989782 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:18.990447 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 04:15:18.991047 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":450,"threads_started":1}
I20260807 04:15:18.994904 10243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0bd8994a054e588480ba5c6d065140" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44811 } }
I20260807 04:15:18.995561 10243 leader_election.cc:302] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f0bd8994a054e588480ba5c6d065140; no voters:
I20260807 04:15:18.996312 10243 leader_election.cc:288] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:18.996484 10248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:18.997013 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:18.998762 10243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:19.008134 10248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [term 1 LEADER]: Becoming Leader. State: Replica: 7f0bd8994a054e588480ba5c6d065140, State: Running, Role: LEADER
I20260807 04:15:19.008935 10248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0bd8994a054e588480ba5c6d065140" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44811 } }
I20260807 04:15:19.012526 10249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f0bd8994a054e588480ba5c6d065140" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0bd8994a054e588480ba5c6d065140" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44811 } } }
I20260807 04:15:19.012775 10250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f0bd8994a054e588480ba5c6d065140. Latest consensus state: current_term: 1 leader_uuid: "7f0bd8994a054e588480ba5c6d065140" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f0bd8994a054e588480ba5c6d065140" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44811 } } }
I20260807 04:15:19.013075 10249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:19.013139 10250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:19.020735 10268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:19.026748 10268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:19.031898 10246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:19.010056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36315 (local address 127.18.193.62:44811)
0807 04:15:19.010253 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:19.010260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:19.010949 (+ 689us) server_negotiation.cc:407] Connection header received
0807 04:15:19.011252 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:19.011262 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:19.011373 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:19.011458 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:19.014161 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:19.015766 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:19.018482 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:19.019576 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:19.019765 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:19.023472 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:19.023484 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:19.023490 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:19.023536 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:19.028191 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:19.029224 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:19.029230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:19.029234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:19.029412 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:19.029906 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:19.029912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:19.029915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:19.030464 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:19.030757 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:19.031208 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 04:15:19.031475 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:15:19.037850 10268 catalog_manager.cc:1347] Generated new cluster ID: 58e569e66a1b40b388388c424cd3109c
I20260807 04:15:19.038000 10268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:19.067291 10268 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:19.068783 10268 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:19.081632 10268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f0bd8994a054e588480ba5c6d065140: Generated new TSK 0
I20260807 04:15:19.082501 10268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:19.098933 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10174
I20260807 04:15:19.100544 10238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:19.197949 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10174
2026-08-07T04:15:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:19 dist-test-slave-8n49 krb5kdc[10277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:19 dist-test-slave-8n49 krb5kdc[10277](info): set up 2 sockets
Aug 07 04:15:19 dist-test-slave-8n49 krb5kdc[10277](info): commencing operation
krb5kdc: starting...
W20260807 04:15:20.367151 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.004s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:20 dist-test-slave-8n49 krb5kdc[10277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:20.685351 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:45033
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:45033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:21.031625 10290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:21.031991 10290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:21.032171 10290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:21.039660 10290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:21.039825 10290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:21.039932 10290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:21.040057 10290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:21.040138 10290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:21.050029 10290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:45033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:21.051400 10290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:21.052765 10290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:21.064324 10298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:21.064366 10301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:21.064392 10299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:21.065618 10290 server_base.cc:1034] running on GCE node
I20260807 04:15:21.066466 10290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:21.067850 10290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:21.069046 10290 hybrid_clock.cc:648] HybridClock initialized: now 1786076121068976 us; error 60 us; skew 500 ppm
I20260807 04:15:21.069350 10290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:21 dist-test-slave-8n49 krb5kdc[10277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076121, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:21.077925 10290 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:21.079505 10290 webserver.cc:460] Webserver started at http://127.18.193.62:38341/ using document root <none> and password file <none>
I20260807 04:15:21.080224 10290 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:21.080330 10290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:21.080669 10290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:21.083688 10290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "8fe87ce29a3446f7883578bc4f2e6ed2"
format_stamp: "Formatted at 2026-08-07 04:15:21 on dist-test-slave-8n49"
server_key: "31196167675e96d8a501b7eacddfe97d"
server_key_iv: "2bd165e5c47c3e3da1b93d6d95b1fb62"
server_key_version: "encryptionkey@0"
I20260807 04:15:21.084434 10290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "8fe87ce29a3446f7883578bc4f2e6ed2"
format_stamp: "Formatted at 2026-08-07 04:15:21 on dist-test-slave-8n49"
server_key: "31196167675e96d8a501b7eacddfe97d"
server_key_iv: "2bd165e5c47c3e3da1b93d6d95b1fb62"
server_key_version: "encryptionkey@0"
I20260807 04:15:21.089689 10290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:15:21.093232 10309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:21.094615 10290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:15:21.094762 10290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "8fe87ce29a3446f7883578bc4f2e6ed2"
format_stamp: "Formatted at 2026-08-07 04:15:21 on dist-test-slave-8n49"
server_key: "31196167675e96d8a501b7eacddfe97d"
server_key_iv: "2bd165e5c47c3e3da1b93d6d95b1fb62"
server_key_version: "encryptionkey@0"
I20260807 04:15:21.094929 10290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:21.116873 10290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:21.121089 10290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:21.121345 10290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:21.141189 10290 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:45033
I20260807 04:15:21.141220 10362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:45033 every 8 connection(s)
I20260807 04:15:21.142620 10290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:21.146524 10363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:21.146823 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10290
I20260807 04:15:21.147027 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:21.147594 19204 external_mini_cluster.cc:1428] Setting key 1b334b4d4d74bcf28f2b9dc0e7f5c357
I20260807 04:15:21.160161 10363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2: Bootstrap starting.
I20260807 04:15:21.161252 10366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:21.149561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51397 (local address 127.18.193.62:45033)
0807 04:15:21.150451 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:21.150463 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:21.150507 (+ 44us) server_negotiation.cc:407] Connection header received
0807 04:15:21.152068 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:21.152091 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:21.152514 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:21.152994 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:21.153575 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:21.154975 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:21.156300 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:21.157084 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:21.157315 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:21.158784 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:21.158791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:21.158804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:21.159350 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:21.159425 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:21.159450 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:21.159579 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:21.159867 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 04:15:21.160132 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":382,"threads_started":1}
I20260807 04:15:21.163522 10363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:21.164721 10363 log.cc:824] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:21.165053 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:21.168160 10363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2: No bootstrap required, opened a new log
I20260807 04:15:21.177323 10366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:21.170243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53861 (local address 127.18.193.62:45033)
0807 04:15:21.170474 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:21.170484 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:21.170972 (+ 488us) server_negotiation.cc:407] Connection header received
0807 04:15:21.171206 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:21.171210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:21.171271 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:21.171356 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:21.172046 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:21.172853 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:21.174744 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:21.175222 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:21.175324 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:21.176492 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:21.176497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:21.176500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:21.176520 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:21.176617 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:21.176636 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:21.176713 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:21.176879 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 04:15:21.176953 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:15:21.178881 10363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45033 } }
I20260807 04:15:21.179284 10363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:21.179405 10363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe87ce29a3446f7883578bc4f2e6ed2, State: Initialized, Role: FOLLOWER
I20260807 04:15:21.180049 10363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45033 } }
I20260807 04:15:21.180320 10363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:21.180490 10363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:21.180744 10363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:21.183054 10363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45033 } }
I20260807 04:15:21.183581 10363 leader_election.cc:302] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fe87ce29a3446f7883578bc4f2e6ed2; no voters:
I20260807 04:15:21.184177 10363 leader_election.cc:288] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:21.184376 10373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:21.186457 10363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:21.186875 10373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe87ce29a3446f7883578bc4f2e6ed2, State: Running, Role: LEADER
I20260807 04:15:21.187426 10373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45033 } }
I20260807 04:15:21.190625 10375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe87ce29a3446f7883578bc4f2e6ed2. Latest consensus state: current_term: 1 leader_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45033 } } }
I20260807 04:15:21.190550 10374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe87ce29a3446f7883578bc4f2e6ed2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 45033 } } }
I20260807 04:15:21.191018 10375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:21.191035 10374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:21.193396 10383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:21.199983 10383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:21.209980 10383 catalog_manager.cc:1347] Generated new cluster ID: 664960b9641c4ef79d96530725d0d8c4
I20260807 04:15:21.210147 10383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:21.228253 10383 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:21.230145 10383 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:21.237298 10383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fe87ce29a3446f7883578bc4f2e6ed2: Generated new TSK 0
I20260807 04:15:21.238312 10383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:21.246196 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10290
W20260807 04:15:21.247620 10358 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51397
W20260807 04:15:21.248314 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10290: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:21.248476 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10290
2026-08-07T04:15:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:21 dist-test-slave-8n49 krb5kdc[10395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:21 dist-test-slave-8n49 krb5kdc[10395](info): set up 2 sockets
Aug 07 04:15:21 dist-test-slave-8n49 krb5kdc[10395](info): commencing operation
krb5kdc: starting...
W20260807 04:15:22.426239 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:22 dist-test-slave-8n49 krb5kdc[10395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:22.749869 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:46263
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:46263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:23.111207 10408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:23.111563 10408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:23.111761 10408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:23.120077 10408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:23.120205 10408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:23.120302 10408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:23.120426 10408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:23.120528 10408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:23.131495 10408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:46263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:46263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:23.132968 10408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:23.134303 10408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:23.146380 10413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:23.146514 10416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:23.146358 10414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:23.147572 10408 server_base.cc:1034] running on GCE node
I20260807 04:15:23.148397 10408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:23.149852 10408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:23.151090 10408 hybrid_clock.cc:648] HybridClock initialized: now 1786076123151042 us; error 43 us; skew 500 ppm
I20260807 04:15:23.151446 10408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:23 dist-test-slave-8n49 krb5kdc[10395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076123, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:23.154644 10408 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:23.156234 10408 webserver.cc:460] Webserver started at http://127.18.193.62:39053/ using document root <none> and password file <none>
I20260807 04:15:23.157009 10408 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:23.157162 10408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:23.157471 10408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:23.160473 10408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "b25cf9efaf81450894838c8779a842a2"
format_stamp: "Formatted at 2026-08-07 04:15:23 on dist-test-slave-8n49"
server_key: "76b7d31000c53e97dfa745536a041e08"
server_key_iv: "4ed3a1f9c7c926d4dea88a3c2b245101"
server_key_version: "encryptionkey@0"
I20260807 04:15:23.161329 10408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "b25cf9efaf81450894838c8779a842a2"
format_stamp: "Formatted at 2026-08-07 04:15:23 on dist-test-slave-8n49"
server_key: "76b7d31000c53e97dfa745536a041e08"
server_key_iv: "4ed3a1f9c7c926d4dea88a3c2b245101"
server_key_version: "encryptionkey@0"
I20260807 04:15:23.166615 10408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:15:23.170331 10423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:23.171782 10408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:15:23.172005 10408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "b25cf9efaf81450894838c8779a842a2"
format_stamp: "Formatted at 2026-08-07 04:15:23 on dist-test-slave-8n49"
server_key: "76b7d31000c53e97dfa745536a041e08"
server_key_iv: "4ed3a1f9c7c926d4dea88a3c2b245101"
server_key_version: "encryptionkey@0"
I20260807 04:15:23.172231 10408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:23.178217 10408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:23.182267 10408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:23.182607 10408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:23.201630 10408 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:46263
I20260807 04:15:23.201674 10479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:46263 every 8 connection(s)
I20260807 04:15:23.203091 10408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:23.206847 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:23.212601 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10408
I20260807 04:15:23.212788 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:23.213238 19204 external_mini_cluster.cc:1428] Setting key 5c9df93a2aef14bdf58d6f79402e3422
I20260807 04:15:23.218150 10480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2: Bootstrap starting.
I20260807 04:15:23.221549 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:23.215260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46317 (local address 127.18.193.62:46263)
0807 04:15:23.216214 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:23.216226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:23.216268 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:15:23.217997 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:23.218020 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:23.218404 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:23.218731 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:23.219941 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:23.219949 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:23.219963 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:23.219983 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:23.220048 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:23.220070 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:23.220277 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:23.220382 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 04:15:23.220663 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":469,"threads_started":1}
I20260807 04:15:23.222091 10480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:23.223367 10480 log.cc:824] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:23.225749 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:23.226737 10480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2: No bootstrap required, opened a new log
I20260807 04:15:23.233474 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:23.230960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40335 (local address 127.18.193.62:46263)
0807 04:15:23.231147 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:23.231153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:23.231505 (+ 352us) server_negotiation.cc:407] Connection header received
0807 04:15:23.231647 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:23.231652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:23.231711 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:23.231815 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:23.232813 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:23.232819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:23.232824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:15:23.232842 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:23.232894 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:15:23.232915 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:23.233054 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:23.233124 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 04:15:23.233188 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:15:23.236868 10480 raft_consensus.cc:348] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25cf9efaf81450894838c8779a842a2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46263 } }
I20260807 04:15:23.237223 10480 raft_consensus.cc:374] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:15:23.237299 10480 raft_consensus.cc:729] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b25cf9efaf81450894838c8779a842a2, State: Initialized, Role: FOLLOWER
I20260807 04:15:23.237905 10480 consensus_queue.cc:260] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25cf9efaf81450894838c8779a842a2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46263 } }
I20260807 04:15:23.238137 10480 raft_consensus.cc:388] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:15:23.238240 10480 raft_consensus.cc:482] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:15:23.238371 10480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:23.241082 10480 raft_consensus.cc:504] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25cf9efaf81450894838c8779a842a2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46263 } }
I20260807 04:15:23.241544 10480 leader_election.cc:302] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b25cf9efaf81450894838c8779a842a2; no voters:
I20260807 04:15:23.242175 10480 leader_election.cc:288] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:15:23.242329 10490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:23.243505 10490 raft_consensus.cc:686] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [term 1 LEADER]: Becoming Leader. State: Replica: b25cf9efaf81450894838c8779a842a2, State: Running, Role: LEADER
I20260807 04:15:23.244067 10490 consensus_queue.cc:237] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25cf9efaf81450894838c8779a842a2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46263 } }
I20260807 04:15:23.244521 10480 sys_catalog.cc:564] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:23.246757 10492 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b25cf9efaf81450894838c8779a842a2. Latest consensus state: current_term: 1 leader_uuid: "b25cf9efaf81450894838c8779a842a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25cf9efaf81450894838c8779a842a2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46263 } } }
I20260807 04:15:23.247195 10492 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:23.247730 10491 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b25cf9efaf81450894838c8779a842a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25cf9efaf81450894838c8779a842a2" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 46263 } } }
I20260807 04:15:23.248060 10491 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:23.253353 10497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:23.259781 10497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:23.268733 10497 catalog_manager.cc:1347] Generated new cluster ID: 0047c79503d74c3e8032807564f0fa34
I20260807 04:15:23.268863 10497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:23.299183 10497 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:23.299721 10497 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:23.319243 10497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b25cf9efaf81450894838c8779a842a2: Generated new TSK 0
I20260807 04:15:23.320420 10497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:23.329070 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10408
W20260807 04:15:23.330200 10475 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46317
W20260807 04:15:23.330961 19204 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10408: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:15:23.331127 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10408
2026-08-07T04:15:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2082 ms)
[----------] 18 tests from AuthTokenIssuingTest (37328 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:23 dist-test-slave-8n49 krb5kdc[10514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:23 dist-test-slave-8n49 krb5kdc[10514](info): set up 2 sockets
Aug 07 04:15:23 dist-test-slave-8n49 krb5kdc[10514](info): commencing operation
krb5kdc: starting...
W20260807 04:15:24.502460 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:24 dist-test-slave-8n49 krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:24.826653 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44531
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:39589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:44531,127.18.193.61:35229,127.18.193.60:36127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:25.172147 10528 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:15:25.172581 10528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:25.172717 10528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:25.172863 10528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:25.180142 10528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:25.180270 10528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:25.180377 10528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:25.180469 10528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:25.180588 10528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:25.190402 10528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:44531,127.18.193.61:35229,127.18.193.60:36127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:44531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:25.191771 10528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:25.193151 10528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:25.205025 10533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:25.205123 10536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:25.204991 10534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:25.206300 10528 server_base.cc:1034] running on GCE node
I20260807 04:15:25.207088 10528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:25.208596 10528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:25.209854 10528 hybrid_clock.cc:648] HybridClock initialized: now 1786076125209784 us; error 46 us; skew 500 ppm
I20260807 04:15:25.210191 10528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:25 dist-test-slave-8n49 krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:25.213346 10528 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:25.215179 10528 webserver.cc:460] Webserver started at http://127.18.193.62:41971/ using document root <none> and password file <none>
I20260807 04:15:25.215930 10528 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:25.216023 10528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:25.216289 10528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:25.219502 10528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "01d08770236542d9811c36d9e950ee0f"
format_stamp: "Formatted at 2026-08-07 04:15:25 on dist-test-slave-8n49"
server_key: "3acaca7157a87447f5b4be841b05381d"
server_key_iv: "3fdad2e3080b849b1d03f1d2b11e48ac"
server_key_version: "encryptionkey@0"
I20260807 04:15:25.220268 10528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "01d08770236542d9811c36d9e950ee0f"
format_stamp: "Formatted at 2026-08-07 04:15:25 on dist-test-slave-8n49"
server_key: "3acaca7157a87447f5b4be841b05381d"
server_key_iv: "3fdad2e3080b849b1d03f1d2b11e48ac"
server_key_version: "encryptionkey@0"
I20260807 04:15:25.225464 10528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 04:15:25.228991 10543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:25.230602 10528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:15:25.230803 10528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "01d08770236542d9811c36d9e950ee0f"
format_stamp: "Formatted at 2026-08-07 04:15:25 on dist-test-slave-8n49"
server_key: "3acaca7157a87447f5b4be841b05381d"
server_key_iv: "3fdad2e3080b849b1d03f1d2b11e48ac"
server_key_version: "encryptionkey@0"
I20260807 04:15:25.231048 10528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:25.242923 10528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:25.247449 10528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:25.247756 10528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:25.268126 10528 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:44531
I20260807 04:15:25.268141 10599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:44531 every 8 connection(s)
I20260807 04:15:25.269582 10528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:25.273531 10600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:25.277783 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10528
I20260807 04:15:25.278053 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:25.278721 19204 external_mini_cluster.cc:1428] Setting key 10e0e05b7d825e6ddf9e94ae312f1237
I20260807 04:15:25.280225 10600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:25 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:15:25.304008 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.283466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44531 (local address 127.0.0.1:36418)
0807 04:15:25.284614 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.284649 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:25.286446 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:25.287254 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:25.287260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:25.287961 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:25.288284 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.288301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.289972 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.289977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.291322 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.291340 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.292399 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.292407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.292671 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.293375 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:25.293400 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:25.295571 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:25.300020 (+ 4449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.300030 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.300039 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.300428 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.300931 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.300934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.300936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.301085 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.302109 (+ 1024us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:25.302115 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:25.302330 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 04:15:25.302547 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 04:15:25.302949 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":59,"thread_start_us":692,"threads_started":1}
I20260807 04:15:25.304050 10602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.283465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.18.193.62:44531)
0807 04:15:25.284283 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:25.284299 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:25.284779 (+ 480us) server_negotiation.cc:407] Connection header received
0807 04:15:25.286803 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:25.286808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:25.287025 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:25.287230 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:25.288512 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.289792 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.291505 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.292139 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.292335 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.295823 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:25.295835 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:25.295853 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:25.295894 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:25.299878 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.300600 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.300611 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.300617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.300809 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.301200 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.301208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.301214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.301959 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:25.302162 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:25.302756 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 04:15:25.303134 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":557,"thread_start_us":356,"threads_started":1}
I20260807 04:15:25.307489 10600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:15:25.308908 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.308328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54746)
0807 04:15:25.308583 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.308737 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 04:15:25.309227 10547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
I20260807 04:15:25.310993 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.310514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54756)
0807 04:15:25.310709 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.310810 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.18.193.61@KRBTEST.COM; defaulting to no policy
W20260807 04:15:25.311218 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:35229: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab.
Entry for principal kudu/127.18.193.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab.
I20260807 04:15:25.351408 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:35229
--webserver_interface=127.18.193.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab
--principal=kudu/127.18.193.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:39589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:44531,127.18.193.61:35229,127.18.193.60:36127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:15:25.360599 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } attempt: 1
I20260807 04:15:25.361793 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.361343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54760)
0807 04:15:25.361541 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.361663 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 04:15:25.362835 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.362453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54776)
0807 04:15:25.362619 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.362685 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 04:15:25.363015 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:35229: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
I20260807 04:15:25.431281 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } attempt: 2
I20260807 04:15:25.432482 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.432010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54782)
0807 04:15:25.432215 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.432313 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 04:15:25.433642 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.433256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54784)
0807 04:15:25.433438 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.433518 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 04:15:25.433861 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:35229: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
I20260807 04:15:25.525137 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } attempt: 3
I20260807 04:15:25.526299 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.525860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54788)
0807 04:15:25.526067 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.526166 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 04:15:25.527338 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.526986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54794)
0807 04:15:25.527148 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.527211 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 04:15:25.527529 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:35229: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
I20260807 04:15:25.670765 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } attempt: 4
I20260807 04:15:25.671979 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.671430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54800)
0807 04:15:25.671749 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.671836 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260807 04:15:25.673079 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.672735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54802)
0807 04:15:25.672884 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.672964 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:15:25.673239 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:35229: Network error: Client connection negotiation failed: client connection to 127.18.193.61:35229: connect: Connection refused (error 111)
W20260807 04:15:25.695325 10608 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:15:25.695720 10608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:25.695855 10608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:25.696007 10608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:25.703477 10608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:25.703603 10608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:25.703733 10608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:25.703823 10608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:25.703938 10608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:25.713918 10608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:44531,127.18.193.61:35229,127.18.193.60:36127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab
--principal=kudu/127.18.193.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:35229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.193.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:25.715351 10608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:25.716717 10608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:25.728775 10616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:25.728754 10614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:25.729317 10613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:25.729585 10608 server_base.cc:1034] running on GCE node
I20260807 04:15:25.730374 10608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:25.731755 10608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:25.732995 10608 hybrid_clock.cc:648] HybridClock initialized: now 1786076125732912 us; error 49 us; skew 500 ppm
I20260807 04:15:25.733336 10608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:25 dist-test-slave-8n49 krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:25.736827 10608 init.cc:377] Logged in from keytab as kudu/127.18.193.61@KRBTEST.COM (short username kudu)
I20260807 04:15:25.738399 10608 webserver.cc:460] Webserver started at http://127.18.193.61:36679/ using document root <none> and password file <none>
I20260807 04:15:25.739078 10608 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:25.739171 10608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:25.739437 10608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:25.742635 10608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data/instance:
uuid: "c2af86f14adb48b3b4f0aedb979df10d"
format_stamp: "Formatted at 2026-08-07 04:15:25 on dist-test-slave-8n49"
server_key: "f0b349e6710878111f46e2682b8bc343"
server_key_iv: "b0064193591d49a8183a24ad9f72b11c"
server_key_version: "encryptionkey@0"
I20260807 04:15:25.743532 10608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/wal/instance:
uuid: "c2af86f14adb48b3b4f0aedb979df10d"
format_stamp: "Formatted at 2026-08-07 04:15:25 on dist-test-slave-8n49"
server_key: "f0b349e6710878111f46e2682b8bc343"
server_key_iv: "b0064193591d49a8183a24ad9f72b11c"
server_key_version: "encryptionkey@0"
I20260807 04:15:25.748893 10608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 04:15:25.752534 10623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:25.754006 10608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:15:25.754204 10608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/wal
uuid: "c2af86f14adb48b3b4f0aedb979df10d"
format_stamp: "Formatted at 2026-08-07 04:15:25 on dist-test-slave-8n49"
server_key: "f0b349e6710878111f46e2682b8bc343"
server_key_iv: "b0064193591d49a8183a24ad9f72b11c"
server_key_version: "encryptionkey@0"
I20260807 04:15:25.754432 10608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:25.769861 10608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:25.774326 10608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:25.774616 10608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:25.794137 10608 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.61:35229
I20260807 04:15:25.794224 10676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.61:35229 every 8 connection(s)
I20260807 04:15:25.795518 10608 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
I20260807 04:15:25.799280 10677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:25.804596 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10608
I20260807 04:15:25.804805 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-1/wal/instance
I20260807 04:15:25.805336 19204 external_mini_cluster.cc:1428] Setting key da9963cc5b22523b356cc84201a1e969
I20260807 04:15:25.806797 10677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:25 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:15:25.829718 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.810313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36432 (local address 127.18.193.62:44531)
0807 04:15:25.810883 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:25.810891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:25.811882 (+ 991us) server_negotiation.cc:407] Connection header received
0807 04:15:25.813905 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:25.813914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:25.814013 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:25.814154 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:25.815678 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.816943 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.818765 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.819198 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.819304 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.823587 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:25.823592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:25.823595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:25.823641 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:25.826449 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.827237 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.827249 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.827253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.827376 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.827844 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.827849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.827855 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.828145 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:25.828311 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:25.829088 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 04:15:25.829310 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":285,"threads_started":1}
I20260807 04:15:25.830729 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.810880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44531 (local address 127.0.0.1:36432)
0807 04:15:25.811683 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.811742 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:25.813623 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:25.814344 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:25.814351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:25.814887 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:25.815453 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.815471 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.817106 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.817111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.818586 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.818606 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.819443 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.819453 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.819797 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.820597 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:25.820634 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:25.823340 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:25.826601 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.826610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.826628 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.827103 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.827528 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.827533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.827537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.827723 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.828288 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:25.828297 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:25.828717 (+ 420us) client_negotiation.cc:769] Sending connection context
0807 04:15:25.829008 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 04:15:25.829551 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":343,"threads_started":1}
I20260807 04:15:25.832687 10677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.193.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.60@KRBTEST.COM" created.
Aug 07 04:15:25 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:15:25.848519 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.833671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54818)
0807 04:15:25.833877 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.833897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:25.834015 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:25.835102 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:25.835106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:25.835505 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:25.835643 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.835651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.837637 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.837641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.838519 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.838530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.839397 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.839469 (+ 72us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.839606 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.840238 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:25.840260 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:25.842761 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:25.846043 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.846050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.846054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.846408 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.846774 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.846780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.846783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.846932 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.847521 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:25.847525 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:25.847676 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 04:15:25.847826 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 04:15:25.848122 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 04:15:25.848644 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.833742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54818 (local address 127.18.193.61:35229)
0807 04:15:25.834569 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:25.834580 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:25.834612 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:15:25.834671 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:25.834679 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:25.834916 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:25.835100 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:25.835827 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.837503 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.838661 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.839217 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.839324 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.842992 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:25.843004 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:25.843017 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:25.843044 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:25.845912 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.846538 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.846550 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.846552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.846660 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.847034 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.847041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.847044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.847394 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:25.847627 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:25.847970 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 04:15:25.848237 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":595,"thread_start_us":442,"threads_started":1}
I20260807 04:15:25.851442 10677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:15:25.852731 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.852124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57328)
0807 04:15:25.852330 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.852499 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 04:15:25.853011 10625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
I20260807 04:15:25.854681 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.854247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57342)
0807 04:15:25.854433 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.854536 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 04:15:25.854924 10677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab.
Entry for principal kudu/127.18.193.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab.
I20260807 04:15:25.876399 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:36127
--webserver_interface=127.18.193.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab
--principal=kudu/127.18.193.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:39589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:44531,127.18.193.61:35229,127.18.193.60:36127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:15:25.904271 10677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 1
I20260807 04:15:25.905619 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.905122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57346)
0807 04:15:25.905345 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.905462 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 04:15:25.906723 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.906352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57354)
0807 04:15:25.906532 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.906611 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 04:15:25.906992 10677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
I20260807 04:15:25.972498 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } attempt: 5
I20260807 04:15:25.975250 10677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 2
I20260807 04:15:25.976267 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.975796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57360)
0807 04:15:25.975982 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.976078 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 04:15:25.977231 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.976903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57370)
0807 04:15:25.977045 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.977120 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:15:25.977494 10677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Aug 07 04:15:25 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:15:25.987025 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.973348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:54830)
0807 04:15:25.973611 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.973636 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:25.973796 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:25.974200 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:25.974204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:25.974714 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:25.974959 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.974973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.976264 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.976270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.977624 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:25.977650 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.978314 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.978320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:25.978529 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.979156 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:25.979186 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:25.981584 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:25.984504 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.984510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.984514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.984937 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.985314 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:25.985319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:25.985323 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:25.985483 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:25.986025 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:25.986030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:25.986196 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 04:15:25.986360 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 04:15:25.986650 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 04:15:25.987020 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.973565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.18.193.61:35229)
0807 04:15:25.973758 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:25.973762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:25.973782 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:15:25.973955 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:25.973959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:25.974041 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:25.974123 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:25.975109 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.976116 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.977766 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:25.978164 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:25.978265 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:25.981779 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:25.981785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:25.981788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:25.981818 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:25.984358 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.985054 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.985061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.985064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.985196 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:25.985602 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:25.985607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:25.985610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:25.985890 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:25.986068 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:25.986500 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 04:15:25.986711 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:15:25.988066 10600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:15:25.989149 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.988764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57372)
0807 04:15:25.988921 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.989014 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 04:15:25.990061 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:25.989725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57386)
0807 04:15:25.989862 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:25.989968 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 04:15:25.990294 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
I20260807 04:15:26.041566 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 1
I20260807 04:15:26.042850 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.042305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57394)
0807 04:15:26.042573 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.042675 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 04:15:26.043850 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.043523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57398)
0807 04:15:26.043679 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.043748 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:15:26.044114 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
I20260807 04:15:26.068756 10677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 3
I20260807 04:15:26.069897 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.069429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57404)
0807 04:15:26.069635 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.069740 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 04:15:26.071025 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.070610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57406)
0807 04:15:26.070840 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.070912 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260807 04:15:26.071226 10677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
I20260807 04:15:26.112408 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 2
I20260807 04:15:26.113844 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.113330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57410)
0807 04:15:26.113583 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.113676 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 04:15:26.114848 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.114499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57418)
0807 04:15:26.114650 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.114736 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 04:15:26.115059 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
I20260807 04:15:26.214455 10677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 4
I20260807 04:15:26.215732 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.215187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57426)
0807 04:15:26.215467 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.215561 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 04:15:26.216786 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.216428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57440)
0807 04:15:26.216604 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.216681 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 04:15:26.216986 10677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
I20260807 04:15:26.221266 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 3
I20260807 04:15:26.222240 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.221803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57448)
0807 04:15:26.222004 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.222089 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 04:15:26.223224 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.222836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:57460)
0807 04:15:26.223046 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.223123 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 04:15:26.223449 10600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:36127: Network error: Client connection negotiation failed: client connection to 127.18.193.60:36127: connect: Connection refused (error 111)
W20260807 04:15:26.226140 10685 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:15:26.226495 10685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:26.226624 10685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:26.226763 10685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:26.234200 10685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:26.234326 10685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:26.234445 10685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:26.234532 10685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:26.234632 10685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:26.244518 10685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:39589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:44531,127.18.193.61:35229,127.18.193.60:36127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab
--principal=kudu/127.18.193.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:36127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.193.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:26.245908 10685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:26.247193 10685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:26.259089 10691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:26.259179 10692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:26.259752 10694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:26.260761 10685 server_base.cc:1034] running on GCE node
I20260807 04:15:26.261579 10685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:26.263057 10685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:26.264320 10685 hybrid_clock.cc:648] HybridClock initialized: now 1786076126264247 us; error 48 us; skew 500 ppm
I20260807 04:15:26.264773 10685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076126, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:26.271368 10685 init.cc:377] Logged in from keytab as kudu/127.18.193.60@KRBTEST.COM (short username kudu)
I20260807 04:15:26.273155 10685 webserver.cc:460] Webserver started at http://127.18.193.60:41425/ using document root <none> and password file <none>
I20260807 04:15:26.273861 10685 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:26.273953 10685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:26.274303 10685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:26.277427 10685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data/instance:
uuid: "0b0af3a6154e4a02a7e5b98c29ae8351"
format_stamp: "Formatted at 2026-08-07 04:15:26 on dist-test-slave-8n49"
server_key: "549d49f4cc673d81fde254d4db55c2b2"
server_key_iv: "eb006f4cefffac82112eb23bf5d04fe4"
server_key_version: "encryptionkey@0"
I20260807 04:15:26.278221 10685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/wal/instance:
uuid: "0b0af3a6154e4a02a7e5b98c29ae8351"
format_stamp: "Formatted at 2026-08-07 04:15:26 on dist-test-slave-8n49"
server_key: "549d49f4cc673d81fde254d4db55c2b2"
server_key_iv: "eb006f4cefffac82112eb23bf5d04fe4"
server_key_version: "encryptionkey@0"
I20260807 04:15:26.283341 10685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:15:26.286938 10701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:26.288226 10685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:15:26.288441 10685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/wal
uuid: "0b0af3a6154e4a02a7e5b98c29ae8351"
format_stamp: "Formatted at 2026-08-07 04:15:26 on dist-test-slave-8n49"
server_key: "549d49f4cc673d81fde254d4db55c2b2"
server_key_iv: "eb006f4cefffac82112eb23bf5d04fe4"
server_key_version: "encryptionkey@0"
I20260807 04:15:26.288756 10685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:26.310853 10685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:26.314950 10685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:26.315230 10685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:26.335628 10685 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.60:36127
I20260807 04:15:26.335649 10756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.60:36127 every 8 connection(s)
I20260807 04:15:26.337318 10685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
I20260807 04:15:26.341192 10757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:26.346691 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10685
I20260807 04:15:26.346906 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786075749366013-19204-0/minicluster-data/master-2/wal/instance
I20260807 04:15:26.347362 19204 external_mini_cluster.cc:1428] Setting key 7eb763dee64d17abd7c87efef17fe898
I20260807 04:15:26.347887 10757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076126, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
I20260807 04:15:26.368148 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.350780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.18.193.62:44531)
0807 04:15:26.351301 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.351309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.352212 (+ 903us) server_negotiation.cc:407] Connection header received
0807 04:15:26.353061 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.353067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.353191 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.353311 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.354615 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.355354 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.357237 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.357914 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.358031 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.361824 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.361830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.361833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.361866 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.364917 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.365697 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.365703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.365707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.365839 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.366307 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.366314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.366322 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.366602 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.366774 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.367529 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.367745 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":255,"threads_started":1}
I20260807 04:15:26.369299 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.349342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.18.193.60:36127)
0807 04:15:26.350100 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.350109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.350146 (+ 37us) server_negotiation.cc:407] Connection header received
0807 04:15:26.352734 (+ 2588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.352756 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.352999 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.353161 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.354719 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.356238 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.357740 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.358541 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.358678 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.363041 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.363049 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.363061 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.363094 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.365924 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.366642 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.366648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.366650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.366775 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.367168 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.367179 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.367182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.367533 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.367880 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.368194 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.368450 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":355,"threads_started":1}
I20260807 04:15:26.369341 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.350836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:44531 (local address 127.0.0.1:33240)
0807 04:15:26.351776 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.351838 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:26.352768 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:26.353392 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:26.353398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:26.354084 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:26.354438 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.354454 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.355535 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.355540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.357071 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.357084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.358153 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.358161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.358379 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.359205 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:26.359231 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:26.361617 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:26.365053 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.365071 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.365084 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.365563 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.365961 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.365966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.365970 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.366180 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.366743 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:26.366751 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:26.367080 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 04:15:26.367408 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 04:15:26.367899 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"spinlock_wait_cycles":13568,"thread_start_us":597,"threads_started":1}
I20260807 04:15:26.370929 10757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076126, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:15:26.384492 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.371827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:35229 (local address 127.0.0.1:53224)
0807 04:15:26.372070 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.372084 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:26.372215 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:26.372767 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:26.372771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:26.373213 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:26.373351 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.373359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.374338 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.374345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.375224 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.375235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.375969 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.375974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.376172 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.376771 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:26.376787 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:26.378846 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:26.382110 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.382116 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.382120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.382477 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.382864 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.382870 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.382873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.383027 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.383547 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:26.383552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:26.383702 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 04:15:26.383845 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 04:15:26.384131 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260807 04:15:26.384524 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.371767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.18.193.61:35229)
0807 04:15:26.371971 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.371977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.372157 (+ 180us) server_negotiation.cc:407] Connection header received
0807 04:15:26.372343 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.372347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.372451 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.372551 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.373510 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.374197 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.375382 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.375815 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.375955 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.379048 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.379053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.379056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.379087 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.381977 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.382597 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.382604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.382607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.382747 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.383142 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.383148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.383151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.383427 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.383580 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.383947 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.384150 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:15:26.385825 10757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:15:26.386263 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.372541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33256 (local address 127.18.193.62:44531)
0807 04:15:26.372745 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.372750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.372766 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:15:26.372810 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.372816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.372898 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.373008 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.374160 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.374842 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.376073 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.376645 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.376773 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.380826 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.380831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.380834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.380865 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.383444 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.384077 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.384084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.384087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.384225 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.384837 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.384840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.384842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.385092 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.385389 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.385695 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.385930 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076126, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:15:26.400400 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.386530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:44782)
0807 04:15:26.386705 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.386722 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:26.386831 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:26.387229 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:26.387232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:26.387707 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:26.387885 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.387898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.388983 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.388991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.390429 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.390441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.391358 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.391362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.391620 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.392241 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:26.392263 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:26.395018 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:26.398001 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.398005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.398008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.398343 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.398769 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.398772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.398774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.398900 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.399530 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:26.399558 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:26.399763 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 04:15:26.399914 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 04:15:26.400120 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 04:15:26.400880 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.386774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44782 (local address 127.18.193.60:36127)
0807 04:15:26.386944 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.386951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.386968 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:15:26.387012 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.387019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.387087 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.387192 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.388042 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.388870 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.390570 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.391217 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.391344 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.395206 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.395211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.395214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.395242 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.397824 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.398476 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.398483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.398490 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.398640 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.399017 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.399022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.399025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.399398 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.399573 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.400051 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.400302 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:15:26.400786 10600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 4
I20260807 04:15:26.405339 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.388241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53234 (local address 127.18.193.61:35229)
0807 04:15:26.388405 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.388411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.388428 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:15:26.388604 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.388609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.388676 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.388779 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.390340 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.391391 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.393231 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.393653 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.393749 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.397977 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.397985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.397989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.398026 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.401436 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.402268 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.402274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.402280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.402468 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.402932 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.402936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.402939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.403287 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.403571 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.404764 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.405004 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:15:26.406489 10757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: Bootstrap starting.
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
I20260807 04:15:26.410357 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:26.410666 10757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:26.412133 10757 log.cc:824] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:26.415868 10757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: No bootstrap required, opened a new log
I20260807 04:15:26.413556 10571 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "01d08770236542d9811c36d9e950ee0f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33256
I20260807 04:15:26.417518 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.401479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:44794)
0807 04:15:26.401856 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.401875 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:26.401984 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:26.402361 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:26.402369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:26.402887 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:26.403125 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.403137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.404737 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.404742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.406093 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.406110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.407126 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.407133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.407331 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.407953 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:26.407977 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:26.410701 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:26.414542 (+ 3841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.414548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.414552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.415117 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.415538 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.415544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.415551 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.415740 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.416440 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:26.416445 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:26.416661 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 04:15:26.416835 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 04:15:26.417128 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
I20260807 04:15:26.417541 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.401835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44794 (local address 127.18.193.60:36127)
0807 04:15:26.402036 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.402044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.402062 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:15:26.402113 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.402117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.402196 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.403278 (+ 1082us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.403425 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.404601 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.406282 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.406996 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.407112 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.410891 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.410897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.410901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.410946 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.414393 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.415247 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.415252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.415255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.415417 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.415873 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.415879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.415885 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.416295 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.416480 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.416962 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.417195 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:15:26.423441 10600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f: Bootstrap starting.
I20260807 04:15:26.426630 10757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
I20260807 04:15:26.427089 10757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0af3a6154e4a02a7e5b98c29ae8351, State: Initialized, Role: FOLLOWER
I20260807 04:15:26.427649 10600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:26.427646 10757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
I20260807 04:15:26.428776 10764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.429119 10600 log.cc:824] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:26.429171 10764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:26.430056 10757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:26.432832 10600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f: No bootstrap required, opened a new log
W20260807 04:15:26.440272 10776 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:15:26.440407 10776 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:15:26.440665 10600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
I20260807 04:15:26.441162 10600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01d08770236542d9811c36d9e950ee0f, State: Initialized, Role: FOLLOWER
I20260807 04:15:26.441654 10600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
I20260807 04:15:26.449514 10779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.450290 10779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:26.450627 10600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:26.450711 10571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 04:15:26.450907 10571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:26.438341 10685 master_runner.cc:186] Error getting master registration for 127.18.193.62:44531: OK, instance_id { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" instance_seqno: 1786076125262243 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:15:26.451413 10685 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:15:26.455046 10571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
I20260807 04:15:26.457060 10571 leader_election.cc:288] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [CANDIDATE]: Term 1 election: Requested vote from peers c2af86f14adb48b3b4f0aedb979df10d (127.18.193.61:35229), 0b0af3a6154e4a02a7e5b98c29ae8351 (127.18.193.60:36127)
I20260807 04:15:26.458909 10732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01d08770236542d9811c36d9e950ee0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351"
I20260807 04:15:26.459410 10732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:26.463079 10732 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01d08770236542d9811c36d9e950ee0f in term 1.
I20260807 04:15:26.464069 10545 leader_election.cc:302] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01d08770236542d9811c36d9e950ee0f, 0b0af3a6154e4a02a7e5b98c29ae8351; no voters:
I20260807 04:15:26.464578 10779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:26.479923 10779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [term 1 LEADER]: Becoming Leader. State: Replica: 01d08770236542d9811c36d9e950ee0f, State: Running, Role: LEADER
I20260807 04:15:26.480613 10779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
W20260807 04:15:26.483065 10790 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:15:26.483228 10790 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 04:15:26.500891 10549 tablet.cc:2369] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:15:26.503057 10791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01d08770236542d9811c36d9e950ee0f. Latest consensus state: current_term: 1 leader_uuid: "01d08770236542d9811c36d9e950ee0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.503573 10791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:26.515311 10800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:26.517827 10800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:15:26.506603 10528 master_runner.cc:186] Error getting master registration for 127.18.193.61:35229: OK, instance_id { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" instance_seqno: 1786076125788508 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:15:26.520107 10528 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:15:26.520236 10677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } attempt: 5
Aug 07 04:15:26 dist-test-slave-8n49 krb5kdc[10514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076125, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
I20260807 04:15:26.546361 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.521640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.18.193.60:36127)
0807 04:15:26.521819 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.521825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.521844 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:15:26.521914 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.521932 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.522024 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.522126 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.523276 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.524475 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.526720 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.527482 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.527623 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.535983 (+ 8360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.535990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.535995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.536034 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.539800 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.540912 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.540926 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.540930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.541093 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.543356 (+ 2263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.543361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.543378 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.543741 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.543901 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.545689 (+ 1788us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.545956 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:15:26.550720 10732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [term 1 FOLLOWER]: Refusing update from remote peer 01d08770236542d9811c36d9e950ee0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:15:26.551864 10791 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:15:26.565107 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.520883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:36127 (local address 127.0.0.1:44800)
0807 04:15:26.521285 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:26.521305 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:26.521419 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:26.522318 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:26.522324 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:26.522855 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:26.523086 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.523099 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.524696 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.524701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.526153 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:26.526170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.527845 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.527851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:26.528103 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.529385 (+ 1282us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:26.529411 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:26.534899 (+ 5488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:26.540092 (+ 5193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.540098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.540102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.540724 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.542307 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:26.542359 (+ 52us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:26.542364 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:26.543016 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:26.544173 (+ 1157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:26.544178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:26.544909 (+ 731us) client_negotiation.cc:769] Sending connection context
0807 04:15:26.545518 (+ 609us) client_negotiation.cc:240] Negotiation successful
0807 04:15:26.548385 (+ 2867us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
W20260807 04:15:26.567860 10708 tablet.cc:2369] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:15:26.577872 10677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d: Bootstrap starting.
I20260807 04:15:26.583398 10764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01d08770236542d9811c36d9e950ee0f. Latest consensus state: current_term: 1 leader_uuid: "01d08770236542d9811c36d9e950ee0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.583731 10764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:26.584748 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.511195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.18.193.61:35229)
0807 04:15:26.511390 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.511397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.521880 (+ 10483us) server_negotiation.cc:407] Connection header received
0807 04:15:26.522127 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.522135 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.522220 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.522298 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.539072 (+ 16774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.540703 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.545872 (+ 5169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.546828 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.546961 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.560505 (+ 13544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.560516 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.560524 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.560613 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.565314 (+ 4701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.582174 (+ 16860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.582182 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.582185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.582350 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.582798 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.582803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.582807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.583206 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.583465 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.584092 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.584349 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:15:26.587880 10677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:26.590565 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.501365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.18.193.62:44531)
0807 04:15:26.504051 (+ 2686us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.504058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.521186 (+ 17128us) server_negotiation.cc:407] Connection header received
0807 04:15:26.521407 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.521414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.521512 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.521617 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.534505 (+ 12888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.536155 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.543367 (+ 7212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.544338 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.544475 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.565700 (+ 21225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.565710 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.565719 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.565771 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.576155 (+ 10384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.581424 (+ 5269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.581431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.581437 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.581624 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.588481 (+ 6857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.588487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.588491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.588956 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.589256 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.589827 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.590116 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2553}
I20260807 04:15:26.592944 10677 log.cc:824] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:26.594681 10791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01d08770236542d9811c36d9e950ee0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.595005 10791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:26.598489 10764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01d08770236542d9811c36d9e950ee0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.598773 10764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:26.599072 10805 mvcc.cc:205] Tried to move back new op lower bound from 7315767814294360064 to 7315767814070218752. Current Snapshot: MvccSnapshot[applied={T|T < 7315767814294360064}]
I20260807 04:15:26.614091 10677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d: No bootstrap required, opened a new log
I20260807 04:15:26.616986 10800 catalog_manager.cc:1347] Generated new cluster ID: c2d3938dbaf646c5bbd25c665e435dc7
I20260807 04:15:26.617106 10800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:26.619948 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:26.526487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44804 (local address 127.18.193.60:36127)
0807 04:15:26.547039 (+ 20552us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.547046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.581672 (+ 34626us) server_negotiation.cc:407] Connection header received
0807 04:15:26.584442 (+ 2770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.584449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.584579 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.584655 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.601019 (+ 16364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.602111 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.605294 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:26.605981 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:26.606087 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:26.610775 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:26.610782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:26.610787 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:26.610829 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:26.614776 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.616154 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.616160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.616164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.616339 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:26.617110 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:26.617127 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:26.617130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:26.617495 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:26.617743 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:26.618745 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0807 04:15:26.619160 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20418,"thread_start_us":49579,"threads_started":1}
I20260807 04:15:26.646289 10677 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
I20260807 04:15:26.646862 10677 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2af86f14adb48b3b4f0aedb979df10d, State: Initialized, Role: FOLLOWER
I20260807 04:15:26.647476 10677 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } }
I20260807 04:15:26.648820 10800 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:26.652869 10800 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:26.658149 10811 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.658593 10811 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:26.659868 10677 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:26.660373 10651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01d08770236542d9811c36d9e950ee0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2af86f14adb48b3b4f0aedb979df10d"
I20260807 04:15:26.660300 10650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:26.669206 10650 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [term 1 FOLLOWER]: Refusing update from remote peer 01d08770236542d9811c36d9e950ee0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:15:26.670297 10791 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:15:26.673283 10651 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01d08770236542d9811c36d9e950ee0f in term 1.
I20260807 04:15:26.714653 10800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f: Generated new TSK 0
I20260807 04:15:26.715919 10800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:26.757187 10811 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01d08770236542d9811c36d9e950ee0f. Latest consensus state: current_term: 1 leader_uuid: "01d08770236542d9811c36d9e950ee0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.757586 10811 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:26.758009 10821 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01d08770236542d9811c36d9e950ee0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.758355 10821 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:26.760064 10779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01d08770236542d9811c36d9e950ee0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d08770236542d9811c36d9e950ee0f" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 44531 } } peers { permanent_uuid: "c2af86f14adb48b3b4f0aedb979df10d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 35229 } } peers { permanent_uuid: "0b0af3a6154e4a02a7e5b98c29ae8351" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 36127 } } }
I20260807 04:15:26.760411 10779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d08770236542d9811c36d9e950ee0f [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:26.787400 10830 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260807 04:15:26.852149 10761 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.836673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.18.193.62:44531)
0807 04:15:26.836884 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.836891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.848882 (+ 11991us) server_negotiation.cc:407] Connection header received
0807 04:15:26.850331 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.850338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.850420 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.850495 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.851878 (+ 1383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33276: BlockingRecv error: recv got EOF from 127.0.0.1:33276 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 04:15:26.855566 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.836319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53252 (local address 127.18.193.61:35229)
0807 04:15:26.836522 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.836529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.853497 (+ 16968us) server_negotiation.cc:407] Connection header received
0807 04:15:26.854175 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.854181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.854262 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.854334 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.855323 (+ 989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53252: BlockingRecv error: recv got EOF from 127.0.0.1:53252 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 04:15:26.859223 10760 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.849468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44806 (local address 127.18.193.60:36127)
0807 04:15:26.849633 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.849639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.856582 (+ 6943us) server_negotiation.cc:407] Connection header received
0807 04:15:26.857338 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.857345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.857422 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.857496 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.858952 (+ 1456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44806: BlockingRecv error: recv got EOF from 127.0.0.1:44806 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 04:15:26.867880 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.864777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.18.193.61:35229)
0807 04:15:26.864951 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.864956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.866375 (+ 1419us) server_negotiation.cc:407] Connection header received
0807 04:15:26.866610 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.866616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.866734 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.866850 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.867617 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53254: BlockingRecv error: recv got EOF from 127.0.0.1:53254 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 04:15:26.886730 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.882426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.18.193.61:35229)
0807 04:15:26.882603 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.882609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.885193 (+ 2584us) server_negotiation.cc:407] Connection header received
0807 04:15:26.885396 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.885409 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.885498 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.885605 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.886507 (+ 902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53256: BlockingRecv error: recv got EOF from 127.0.0.1:53256 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 04:15:26.895962 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.893802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.18.193.61:35229)
0807 04:15:26.893982 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.893989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.894505 (+ 516us) server_negotiation.cc:407] Connection header received
0807 04:15:26.894701 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.894707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.894809 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.894917 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.895718 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53270: BlockingRecv error: recv got EOF from 127.0.0.1:53270 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 04:15:26.909126 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.906884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53286 (local address 127.18.193.61:35229)
0807 04:15:26.907126 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.907135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.907649 (+ 514us) server_negotiation.cc:407] Connection header received
0807 04:15:26.907810 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.907816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.907913 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.908020 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.908892 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53286: BlockingRecv error: recv got EOF from 127.0.0.1:53286 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260807 04:15:26.924682 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.922303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.18.193.61:35229)
0807 04:15:26.922492 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.922498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.923024 (+ 526us) server_negotiation.cc:407] Connection header received
0807 04:15:26.923249 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.923258 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.923365 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.923485 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.924384 (+ 899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53300: BlockingRecv error: recv got EOF from 127.0.0.1:53300 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 04:15:26.949087 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.946604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.18.193.61:35229)
0807 04:15:26.946809 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.946816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.947392 (+ 576us) server_negotiation.cc:407] Connection header received
0807 04:15:26.947601 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.947617 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.947730 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.947855 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.948832 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53306: BlockingRecv error: recv got EOF from 127.0.0.1:53306 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 04:15:26.992847 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:26.990326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.18.193.61:35229)
0807 04:15:26.990528 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:26.990535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:26.991085 (+ 550us) server_negotiation.cc:407] Connection header received
0807 04:15:26.991310 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:26.991316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:26.991428 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:26.991548 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:26.992462 (+ 914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53312: BlockingRecv error: recv got EOF from 127.0.0.1:53312 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 04:15:27.062578 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:27.060212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.18.193.61:35229)
0807 04:15:27.060507 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:27.060515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:27.061012 (+ 497us) server_negotiation.cc:407] Connection header received
0807 04:15:27.061277 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:27.061281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:27.061357 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:27.061446 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:27.062347 (+ 901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53328: BlockingRecv error: recv got EOF from 127.0.0.1:53328 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 04:15:27.197021 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:27.194889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.18.193.61:35229)
0807 04:15:27.195114 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:27.195123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:27.195483 (+ 360us) server_negotiation.cc:407] Connection header received
0807 04:15:27.195743 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:27.195747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:27.195824 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:27.195942 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:27.196757 (+ 815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53330: BlockingRecv error: recv got EOF from 127.0.0.1:53330 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:15:27.442418 10776 catalog_manager.cc:1259] Loaded cluster ID: c2d3938dbaf646c5bbd25c665e435dc7
I20260807 04:15:27.442533 10776 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: loading cluster ID for follower catalog manager: success
I20260807 04:15:27.445004 10776 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: acquiring CA information for follower catalog manager: success
I20260807 04:15:27.446714 10776 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b0af3a6154e4a02a7e5b98c29ae8351: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 04:15:27.458470 10682 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:27.456631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.18.193.61:35229)
0807 04:15:27.456852 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:27.456860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:27.456999 (+ 139us) server_negotiation.cc:407] Connection header received
0807 04:15:27.457253 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:27.457257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:27.457331 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:27.457453 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:27.458222 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53338: BlockingRecv error: recv got EOF from 127.0.0.1:53338 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:15:27.792021 10830 catalog_manager.cc:1259] Loaded cluster ID: c2d3938dbaf646c5bbd25c665e435dc7
I20260807 04:15:27.792142 10830 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d: loading cluster ID for follower catalog manager: success
I20260807 04:15:27.795060 10830 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d: acquiring CA information for follower catalog manager: success
I20260807 04:15:27.796958 10830 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2af86f14adb48b3b4f0aedb979df10d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:15:27.985553 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:27.978603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.18.193.61:35229)
0807 04:15:27.979429 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:27.979434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:27.979452 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:15:27.979516 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:27.979519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:27.979660 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:27.979766 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:27.980282 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:27.981066 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:27.982985 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:27.983448 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:27.983545 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:27.984143 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:27.984757 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:27.984884 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:27.985048 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 04:15:27.985124 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":541,"threads_started":1}
I20260807 04:15:28.004048 10906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:27.997661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.18.193.60:36127)
0807 04:15:27.998329 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:27.998334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:27.998351 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:15:27.998441 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:27.998444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:27.998584 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:27.998730 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:27.999179 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:27.999864 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.001829 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:28.002236 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.002334 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:28.002882 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:28.003379 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:28.003485 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:28.003642 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 04:15:28.003711 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":452,"threads_started":1}
I20260807 04:15:28.032235 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:28.011036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53354 (local address 127.18.193.61:35229)
0807 04:15:28.011350 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:28.011360 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:28.023889 (+ 12529us) server_negotiation.cc:407] Connection header received
0807 04:15:28.024191 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:28.024197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:28.024295 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:28.024361 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:28.025370 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:28.026521 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.028740 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:28.029359 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.029465 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:28.031320 (+ 1855us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:28.031463 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:28.031544 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:28.031780 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 04:15:28.031865 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 04:15:28.033722 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:28.010962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33288 (local address 127.18.193.62:44531)
0807 04:15:28.023033 (+ 12071us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:28.023041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:28.023888 (+ 847us) server_negotiation.cc:407] Connection header received
0807 04:15:28.024028 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:28.024033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:28.024289 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:28.025252 (+ 963us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:28.025653 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:28.027123 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.030367 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:28.030970 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.031084 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:28.031896 (+ 812us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:28.032658 (+ 762us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:28.032760 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:28.033324 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 04:15:28.033414 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11912,"thread_start_us":11800,"threads_started":1}
I20260807 04:15:28.037356 10906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:28.024105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.18.193.60:36127)
0807 04:15:28.024284 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:28.024291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:28.030594 (+ 6303us) server_negotiation.cc:407] Connection header received
0807 04:15:28.030794 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:28.030800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:28.030897 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:28.031003 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:28.031799 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:28.032962 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.035068 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:28.035744 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:28.035884 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:28.036475 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:28.036682 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:28.036781 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:28.036973 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 04:15:28.037066 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:15:28.038775 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10528
I20260807 04:15:28.039878 10595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:28.153159 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10528
W20260807 04:15:28.177031 10624 connection.cc:535] server connection from 127.0.0.1:54830 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 04:15:28.177258 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10608
I20260807 04:15:28.178375 10672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:28.285758 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10608
I20260807 04:15:28.308108 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10685
I20260807 04:15:28.309312 10752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:28.422102 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10685
2026-08-07T04:15:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5097 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:15:28 dist-test-slave-8n49 krb5kdc[10920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:15:28 dist-test-slave-8n49 krb5kdc[10920](info): set up 2 sockets
Aug 07 04:15:28 dist-test-slave-8n49 krb5kdc[10920](info): commencing operation
krb5kdc: starting...
W20260807 04:15:29.601058 19204 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/test-user.keytab.
Aug 07 04:15:29 dist-test-slave-8n49 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:15:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:15:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.193.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
Entry for principal kudu/127.18.193.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab.
I20260807 04:15:29.911487 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37367
--webserver_interface=127.18.193.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:37367,127.18.193.61:37179,127.18.193.60:43501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:15:30.246953 10933 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:15:30.247299 10933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:30.247417 10933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:30.247562 10933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:30.255013 10933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:30.255102 10933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:30.255198 10933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:30.255290 10933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:30.255388 10933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:30.265136 10933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:37367,127.18.193.61:37179,127.18.193.60:43501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.62.keytab
--principal=kudu/127.18.193.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.62:37367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.193.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:30.266419 10933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:30.267745 10933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:30.279677 10942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:30.279646 10938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:30.279718 10939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:30.280404 10933 server_base.cc:1034] running on GCE node
I20260807 04:15:30.281227 10933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:30.282554 10933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:30.283800 10933 hybrid_clock.cc:648] HybridClock initialized: now 1786076130283751 us; error 50 us; skew 500 ppm
I20260807 04:15:30.284142 10933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:30 dist-test-slave-8n49 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:30.287425 10933 init.cc:377] Logged in from keytab as kudu/127.18.193.62@KRBTEST.COM (short username kudu)
I20260807 04:15:30.288820 10933 webserver.cc:460] Webserver started at http://127.18.193.62:38331/ using document root <none> and password file <none>
I20260807 04:15:30.289440 10933 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:30.289525 10933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:30.289783 10933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:30.292816 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data/instance:
uuid: "b58ff9a83dc04660b724e20b29612de6"
format_stamp: "Formatted at 2026-08-07 04:15:30 on dist-test-slave-8n49"
server_key: "826f152f256b080fe37332c68fd3a122"
server_key_iv: "e727ace2879a6d529778145c80714212"
server_key_version: "encryptionkey@0"
I20260807 04:15:30.293565 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/wal/instance:
uuid: "b58ff9a83dc04660b724e20b29612de6"
format_stamp: "Formatted at 2026-08-07 04:15:30 on dist-test-slave-8n49"
server_key: "826f152f256b080fe37332c68fd3a122"
server_key_iv: "e727ace2879a6d529778145c80714212"
server_key_version: "encryptionkey@0"
I20260807 04:15:30.298427 10933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:15:30.301811 10949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:30.303103 10933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:15:30.303233 10933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/wal
uuid: "b58ff9a83dc04660b724e20b29612de6"
format_stamp: "Formatted at 2026-08-07 04:15:30 on dist-test-slave-8n49"
server_key: "826f152f256b080fe37332c68fd3a122"
server_key_iv: "e727ace2879a6d529778145c80714212"
server_key_version: "encryptionkey@0"
I20260807 04:15:30.303406 10933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:30.321584 10933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:30.325891 10933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:30.326130 10933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:30.347009 10933 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.62:37367
I20260807 04:15:30.347028 11005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.62:37367 every 8 connection(s)
I20260807 04:15:30.348449 10933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/data/info.pb
I20260807 04:15:30.351516 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 10933
I20260807 04:15:30.351713 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-0/wal/instance
I20260807 04:15:30.352257 19204 external_mini_cluster.cc:1428] Setting key a8453f050f412225c95918eca5f98b08
I20260807 04:15:30.352783 11006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:30.362844 11006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:30 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.193.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.61@KRBTEST.COM" created.
I20260807 04:15:30.386255 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.366881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37367 (local address 127.0.0.1:38752)
0807 04:15:30.367841 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.367891 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:30.370175 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:30.370995 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:30.371001 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:30.371690 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:30.372018 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:30.372035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.373689 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.373694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:30.374729 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:30.374741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.375639 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.375646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:30.375844 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:30.376521 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:30.376548 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:30.378659 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:30.382028 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:30.382039 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:30.382050 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:30.382437 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:30.382840 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:30.382846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:30.382850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:30.383035 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:30.383776 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:30.383783 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:30.383997 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 04:15:30.384301 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 04:15:30.384928 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":177,"thread_start_us":478,"threads_started":1}
I20260807 04:15:30.386274 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.368350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.18.193.62:37367)
0807 04:15:30.369179 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:30.369188 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:30.369246 (+ 58us) server_negotiation.cc:407] Connection header received
0807 04:15:30.370554 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:30.370559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:30.370773 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:30.370991 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:30.372263 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.373533 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.374873 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.375406 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.375602 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:30.378898 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:30.378906 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:30.378917 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:30.378945 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:30.381872 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:30.382570 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:30.382579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:30.382584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:30.382725 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:30.383132 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:30.383140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:30.383143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:30.383627 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:30.383848 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:30.384418 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 04:15:30.384816 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":630,"thread_start_us":455,"threads_started":1}
I20260807 04:15:30.389955 11006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:15:30.391146 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.390654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40778)
0807 04:15:30.390856 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.390991 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 04:15:30.391489 10952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
I20260807 04:15:30.393215 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.392768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40790)
0807 04:15:30.392944 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.393034 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 04:15:30.393537 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:37179: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab.
Entry for principal kudu/127.18.193.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab.
I20260807 04:15:30.422308 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:37179
--webserver_interface=127.18.193.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab
--principal=kudu/127.18.193.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:37367,127.18.193.61:37179,127.18.193.60:43501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:15:30.442965 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } attempt: 1
I20260807 04:15:30.444456 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.443808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40792)
0807 04:15:30.444082 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.444256 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 04:15:30.445729 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.445232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40800)
0807 04:15:30.445413 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.445513 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 04:15:30.445950 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:37179: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
I20260807 04:15:30.514245 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } attempt: 2
I20260807 04:15:30.515520 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.514998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40812)
0807 04:15:30.515228 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.515343 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 04:15:30.516610 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.516170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40814)
0807 04:15:30.516353 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.516434 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 04:15:30.516796 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:37179: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
I20260807 04:15:30.608073 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } attempt: 3
I20260807 04:15:30.609488 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.608866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40822)
0807 04:15:30.609152 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.609286 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 04:15:30.610707 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.610218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40836)
0807 04:15:30.610405 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.610507 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 04:15:30.610935 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:37179: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
I20260807 04:15:30.754220 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } attempt: 4
I20260807 04:15:30.755555 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.754892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40846)
0807 04:15:30.755255 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.755365 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260807 04:15:30.756666 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.756222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40858)
0807 04:15:30.756384 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.756487 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 04:15:30.756855 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.61:37179: Network error: Client connection negotiation failed: client connection to 127.18.193.61:37179: connect: Connection refused (error 111)
W20260807 04:15:30.760466 11014 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:15:30.760836 11014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:30.760914 11014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:30.760998 11014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:30.768100 11014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:30.768177 11014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:30.768296 11014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:30.768381 11014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:30.768499 11014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:30.778193 11014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:37367,127.18.193.61:37179,127.18.193.60:43501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.61.keytab
--principal=kudu/127.18.193.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.61:37179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.193.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:30.779455 11014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:30.780799 11014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:30.792234 11019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:30.792171 11022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:30.792384 11020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:30.793444 11014 server_base.cc:1034] running on GCE node
I20260807 04:15:30.794185 11014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:30.795504 11014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:30.796749 11014 hybrid_clock.cc:648] HybridClock initialized: now 1786076130796697 us; error 34 us; skew 500 ppm
I20260807 04:15:30.797076 11014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:30 dist-test-slave-8n49 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:30.800176 11014 init.cc:377] Logged in from keytab as kudu/127.18.193.61@KRBTEST.COM (short username kudu)
I20260807 04:15:30.801643 11014 webserver.cc:460] Webserver started at http://127.18.193.61:38121/ using document root <none> and password file <none>
I20260807 04:15:30.802323 11014 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:30.802459 11014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:30.802784 11014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:30.805763 11014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data/instance:
uuid: "d7b9011430714346aa668cce9a7e447d"
format_stamp: "Formatted at 2026-08-07 04:15:30 on dist-test-slave-8n49"
server_key: "c2b3f0a79efec3c8aaa3251d6a84b7c5"
server_key_iv: "9441c99042882f3b1b5161321ffaa8eb"
server_key_version: "encryptionkey@0"
I20260807 04:15:30.806560 11014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/wal/instance:
uuid: "d7b9011430714346aa668cce9a7e447d"
format_stamp: "Formatted at 2026-08-07 04:15:30 on dist-test-slave-8n49"
server_key: "c2b3f0a79efec3c8aaa3251d6a84b7c5"
server_key_iv: "9441c99042882f3b1b5161321ffaa8eb"
server_key_version: "encryptionkey@0"
I20260807 04:15:30.811645 11014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:15:30.815064 11030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:30.816409 11014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:15:30.816689 11014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/wal
uuid: "d7b9011430714346aa668cce9a7e447d"
format_stamp: "Formatted at 2026-08-07 04:15:30 on dist-test-slave-8n49"
server_key: "c2b3f0a79efec3c8aaa3251d6a84b7c5"
server_key_iv: "9441c99042882f3b1b5161321ffaa8eb"
server_key_version: "encryptionkey@0"
I20260807 04:15:30.816957 11014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:30.839666 11014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:30.844506 11014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:30.844843 11014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:30.863826 11014 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.61:37179
I20260807 04:15:30.863847 11084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.61:37179 every 8 connection(s)
I20260807 04:15:30.865334 11014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/data/info.pb
I20260807 04:15:30.868963 11085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:30.873142 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 11014
I20260807 04:15:30.873334 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-1/wal/instance
I20260807 04:15:30.873926 19204 external_mini_cluster.cc:1428] Setting key e899da8db4d4e9e280890f3740ae9def
I20260807 04:15:30.876794 11085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:30 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:15:30.899619 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.880032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38756 (local address 127.18.193.62:37367)
0807 04:15:30.880265 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:30.880274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:30.881523 (+ 1249us) server_negotiation.cc:407] Connection header received
0807 04:15:30.884389 (+ 2866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:30.884393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:30.884490 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:30.884668 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:30.886666 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.887347 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.889109 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.889515 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.889658 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:30.893880 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:30.893885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:30.893888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:30.893919 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:30.896630 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:30.897348 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:30.897354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:30.897359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:30.897479 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:30.897910 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:30.897915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:30.897918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:30.898210 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:30.898373 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:30.899068 (+ 695us) server_negotiation.cc:299] Negotiation successful
0807 04:15:30.899290 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:15:30.900656 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.880480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37367 (local address 127.0.0.1:38756)
0807 04:15:30.881316 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.881377 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:30.884073 (+ 2696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:30.884889 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:30.884902 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:30.885696 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:30.886464 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:30.886502 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.887521 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.887530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:30.888937 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:30.888950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.889780 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.889790 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:30.890140 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:30.890880 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:30.890923 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:30.893653 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:30.896762 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:30.896771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:30.896786 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:30.897219 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:30.897598 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:30.897603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:30.897607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:30.897785 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:30.898351 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:30.898359 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:30.898691 (+ 332us) client_negotiation.cc:769] Sending connection context
0807 04:15:30.898952 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 04:15:30.899444 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":299,"threads_started":1}
I20260807 04:15:30.902468 11085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.193.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.193.60@KRBTEST.COM" created.
Aug 07 04:15:30 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:15:30.918496 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.903230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40864)
0807 04:15:30.903394 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.903417 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:30.903533 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:30.904715 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:30.904720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:30.905156 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:30.905284 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:30.905292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.906915 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.906920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:30.907845 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:30.907857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.908848 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.908854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:30.909043 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:30.909631 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:30.909648 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:30.911580 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:30.916046 (+ 4466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:30.916050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:30.916053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:30.916411 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:30.916871 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:30.916874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:30.916876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:30.917030 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:30.917796 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:30.917799 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:30.917925 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 04:15:30.918045 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 04:15:30.918230 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 04:15:30.918995 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.903508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.18.193.61:37179)
0807 04:15:30.904257 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:30.904262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:30.904285 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:15:30.904325 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:30.904332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:30.904499 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:30.904732 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:30.905463 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.906761 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.907998 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:30.908718 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:30.908843 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:30.911803 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:30.911818 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:30.911837 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:30.911963 (+ 126us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:30.915895 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:30.916547 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:30.916595 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:30.916602 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:30.916750 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:30.917140 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:30.917147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:30.917153 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:30.917642 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:30.917834 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:30.918221 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 04:15:30.918512 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":438,"threads_started":1}
I20260807 04:15:30.921885 11085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:15:30.923202 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.922651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56554)
0807 04:15:30.922863 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.923005 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 04:15:30.923518 11034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:30.925272 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.924829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56566)
0807 04:15:30.925011 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.925118 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 04:15:30.925529 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.193.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab.
Entry for principal kudu/127.18.193.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab.
I20260807 04:15:30.947645 19204 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5Tobyc/build/asan/bin/kudu
/tmp/dist-test-task5Tobyc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:43501
--webserver_interface=127.18.193.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab
--principal=kudu/127.18.193.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.193.20:42159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.193.62:37367,127.18.193.61:37179,127.18.193.60:43501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:15:30.974889 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 1
I20260807 04:15:30.978471 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.975843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56570)
0807 04:15:30.978120 (+ 2277us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.978270 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2089}
I20260807 04:15:30.979835 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:30.979302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56586)
0807 04:15:30.979598 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:30.979699 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260807 04:15:30.980139 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:31.048408 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 2
I20260807 04:15:31.049770 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.049264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56596)
0807 04:15:31.049503 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.049601 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 04:15:31.050769 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.050439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56606)
0807 04:15:31.050585 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.050655 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 04:15:31.051033 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:31.056103 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } attempt: 5
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:15:31.070636 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.056819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40868)
0807 04:15:31.057025 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.057048 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:31.057194 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:31.057606 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:31.057611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:31.058113 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:31.058355 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.058368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.059364 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.059369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.060730 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.060746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.061627 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.061633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.061839 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.062503 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:31.062527 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:31.065076 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:31.068132 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.068139 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.068142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.068609 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.068993 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.069001 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.069004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.069141 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.069669 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:31.069673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:31.069834 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 04:15:31.069978 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 04:15:31.070254 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 04:15:31.070633 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.056892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40868 (local address 127.18.193.61:37179)
0807 04:15:31.057066 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.057071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.057110 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:15:31.057352 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.057357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.057445 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.057538 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.058515 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.059223 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.060920 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.061451 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.061573 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.065273 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.065278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.065281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.065312 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.067997 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.068754 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.068758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.068761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.068879 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.069257 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.069262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.069265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.069546 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.069704 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.070099 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.070303 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:15:31.071609 11006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:15:31.072839 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.072430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56622)
0807 04:15:31.072635 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.072713 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 04:15:31.073769 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.073455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56638)
0807 04:15:31.073592 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.073678 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:15:31.073963 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:31.125203 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 1
I20260807 04:15:31.126358 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.125886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56640)
0807 04:15:31.126089 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.126188 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 04:15:31.127344 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.127029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56652)
0807 04:15:31.127174 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.127245 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 04:15:31.127573 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:31.142305 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 3
I20260807 04:15:31.143529 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.143071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56666)
0807 04:15:31.143266 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.143360 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 04:15:31.144728 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.144222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56682)
0807 04:15:31.144453 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.144544 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260807 04:15:31.144958 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:31.195776 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 2
I20260807 04:15:31.197170 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.196608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56696)
0807 04:15:31.196875 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.196988 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 04:15:31.198325 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.197939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56702)
0807 04:15:31.198129 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.198208 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 04:15:31.198589 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:31.288182 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 4
I20260807 04:15:31.289368 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.288871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56716)
0807 04:15:31.289117 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.289203 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 04:15:31.290491 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.290164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56720)
0807 04:15:31.290323 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.290389 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 04:15:31.290704 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
W20260807 04:15:31.293624 11092 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:15:31.293982 11092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:15:31.294119 11092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:15:31.294260 11092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:15:31.301648 11092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:15:31.301769 11092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:15:31.301900 11092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:15:31.301998 11092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:15:31.302111 11092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:15:31.304800 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 3
I20260807 04:15:31.305929 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.305449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56726)
0807 04:15:31.305683 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.305767 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 04:15:31.306941 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.306583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56728)
0807 04:15:31.306768 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.306846 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 04:15:31.307184 11006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.193.60:43501: Network error: Client connection negotiation failed: client connection to 127.18.193.60:43501: connect: Connection refused (error 111)
I20260807 04:15:31.312789 11092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.193.20:42159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.193.62:37367,127.18.193.61:37179,127.18.193.60:43501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/krb5kdc/kudu_127.18.193.60.keytab
--principal=kudu/127.18.193.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.193.60:43501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.193.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:15:31.314177 11092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:15:31.315620 11092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:15:31.327631 11097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:31.327610 11101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:15:31.327735 11099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:15:31.328775 11092 server_base.cc:1034] running on GCE node
I20260807 04:15:31.329584 11092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:15:31.331063 11092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:15:31.332310 11092 hybrid_clock.cc:648] HybridClock initialized: now 1786076131332237 us; error 50 us; skew 500 ppm
I20260807 04:15:31.332708 11092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076131, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:15:31.341403 11092 init.cc:377] Logged in from keytab as kudu/127.18.193.60@KRBTEST.COM (short username kudu)
I20260807 04:15:31.343007 11092 webserver.cc:460] Webserver started at http://127.18.193.60:35965/ using document root <none> and password file <none>
I20260807 04:15:31.343725 11092 fs_manager.cc:359] Metadata directory not provided
I20260807 04:15:31.343859 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:15:31.344187 11092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:15:31.347756 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data/instance:
uuid: "13ac38e8d69c4dbb986993556dc695e5"
format_stamp: "Formatted at 2026-08-07 04:15:31 on dist-test-slave-8n49"
server_key: "c51c9175aa8cf8d313722c8a71b64ac4"
server_key_iv: "dc4eb4148b703601b55814b7e070094a"
server_key_version: "encryptionkey@0"
I20260807 04:15:31.348824 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/wal/instance:
uuid: "13ac38e8d69c4dbb986993556dc695e5"
format_stamp: "Formatted at 2026-08-07 04:15:31 on dist-test-slave-8n49"
server_key: "c51c9175aa8cf8d313722c8a71b64ac4"
server_key_iv: "dc4eb4148b703601b55814b7e070094a"
server_key_version: "encryptionkey@0"
I20260807 04:15:31.355134 11092 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 04:15:31.359208 11110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:15:31.360540 11092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:15:31.360759 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data,/tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/wal
uuid: "13ac38e8d69c4dbb986993556dc695e5"
format_stamp: "Formatted at 2026-08-07 04:15:31 on dist-test-slave-8n49"
server_key: "c51c9175aa8cf8d313722c8a71b64ac4"
server_key_iv: "dc4eb4148b703601b55814b7e070094a"
server_key_version: "encryptionkey@0"
I20260807 04:15:31.360981 11092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:15:31.380110 11092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:15:31.384439 11092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:15:31.384764 11092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:15:31.403913 11092 rpc_server.cc:307] RPC server started. Bound to: 127.18.193.60:43501
I20260807 04:15:31.403935 11163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.193.60:43501 every 8 connection(s)
I20260807 04:15:31.405396 11092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/data/info.pb
I20260807 04:15:31.407351 19204 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5Tobyc/build/asan/bin/kudu as pid 11092
I20260807 04:15:31.407584 19204 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5Tobyc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786075749366013-19204-0/minicluster-data/master-2/wal/instance
I20260807 04:15:31.408341 19204 external_mini_cluster.cc:1428] Setting key ef36bb5f80a6d2f9395806a05b9c60ee
I20260807 04:15:31.411190 11164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:15:31.420195 11164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076131, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
I20260807 04:15:31.433756 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.410355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.18.193.60:43501)
0807 04:15:31.411147 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.411166 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.411225 (+ 59us) server_negotiation.cc:407] Connection header received
0807 04:15:31.413637 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.413662 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.414154 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.414619 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.415914 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.417974 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.419197 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.420279 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.420503 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.425504 (+ 5001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.425512 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.425522 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.425560 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.429136 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.429856 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.429865 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.429921 (+ 56us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.430153 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.430553 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.430558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.430561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.431135 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.431484 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.431908 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.432390 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":365,"threads_started":1}
I20260807 04:15:31.439571 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.423186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.62:37367 (local address 127.0.0.1:38758)
0807 04:15:31.423894 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.423940 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:31.424084 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:31.424957 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:31.424963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:31.425596 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:31.425888 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.425898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.427130 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.427138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.429006 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.429025 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.429818 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.429825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.430048 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.430721 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:31.430743 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:31.432908 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:31.436830 (+ 3922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.436844 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.436853 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.437449 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.437886 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.437891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.437894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.438066 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.438746 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:31.438752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:31.438899 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 04:15:31.439061 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 04:15:31.439280 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"spinlock_wait_cycles":16128,"thread_start_us":339,"threads_started":1}
I20260807 04:15:31.440091 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.423272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38758 (local address 127.18.193.62:37367)
0807 04:15:31.424623 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.424629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.424645 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:15:31.424697 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.424702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.424761 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.424888 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.426054 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.426910 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.429161 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.429604 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.429754 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.433113 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.433120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.433125 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.433166 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.436650 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.437625 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.437629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.437631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.437774 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.438191 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.438197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.438203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.438603 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.438773 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.439222 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.439571 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1148,"thread_start_us":1014,"threads_started":1}
I20260807 04:15:31.441560 11164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.62@KRBTEST.COM
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076131, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:15:31.453715 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.437113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.18.193.62:37367)
0807 04:15:31.437770 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.437775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.437793 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:15:31.437862 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.437868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.437941 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.438067 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.439775 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.440953 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.442378 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.442852 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.443038 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.447623 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.447627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.447631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.447664 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.451002 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.451695 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.451702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.451706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.451834 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.452234 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.452240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.452244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.452541 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.452874 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.453188 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.453356 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":309,"threads_started":1}
I20260807 04:15:31.455880 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.442348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.61:37179 (local address 127.0.0.1:40882)
0807 04:15:31.442626 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.442644 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:31.442755 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:31.443111 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:31.443115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:31.443467 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:31.443595 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.443603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.444591 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.444596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.445466 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.445478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.446450 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.446467 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.446688 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.447270 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:31.447296 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:31.449696 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:31.453022 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.453030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.453037 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.453551 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.454076 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.454081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.454084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.454267 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.454897 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:31.454902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:31.455058 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 04:15:31.455212 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 04:15:31.455513 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260807 04:15:31.455976 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.442410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.18.193.61:37179)
0807 04:15:31.442629 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.442637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.442716 (+ 79us) server_negotiation.cc:407] Connection header received
0807 04:15:31.442885 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.442891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.442966 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.443076 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.443735 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.444412 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.445622 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.446267 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.446401 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.449896 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.449904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.449911 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.449967 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.452885 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.453740 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.453745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.453747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.453947 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.454387 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.454392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.454395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.454747 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.454933 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.455302 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.455513 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 04:15:31.457518 11164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076131, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.60@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.193.61@KRBTEST.COM
I20260807 04:15:31.470844 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.458388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56744)
0807 04:15:31.458512 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.458530 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:31.458629 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:31.458948 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:31.458952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:31.459283 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:31.459417 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.459426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.460356 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.460363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.461261 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.461272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.461964 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.461970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.462156 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.462679 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:31.462694 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:31.464732 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:31.467783 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.467789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.467793 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.468146 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.468509 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.468515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.468518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.468687 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.469748 (+ 1061us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:31.469751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:31.469877 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 04:15:31.470056 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 04:15:31.470409 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":1}
I20260807 04:15:31.470917 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.458320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56744 (local address 127.18.193.60:43501)
0807 04:15:31.458505 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.458513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.458613 (+ 100us) server_negotiation.cc:407] Connection header received
0807 04:15:31.458706 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.458711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.458779 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.458879 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.459569 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.460230 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.461409 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.461797 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.461921 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.464914 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.464920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.464925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.464960 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.467620 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.468263 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.468270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.468273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.468401 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.468790 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.468797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.468800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.469065 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.469220 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.470149 (+ 929us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.470567 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":184,"server-negotiator.queue_time_us":49}
I20260807 04:15:31.472195 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.455563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.18.193.61:37179)
0807 04:15:31.456286 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.456292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.456323 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:15:31.456516 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.456524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.456633 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.456732 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.458351 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.459402 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.460647 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.461074 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.461214 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.465964 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.465969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.465972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.466001 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.469064 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.469718 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.469721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.469723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.469840 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.470281 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.470284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.470286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.470528 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.470777 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.471688 (+ 911us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.471912 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":337,"threads_started":1}
I20260807 04:15:31.476707 11164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: Bootstrap starting.
I20260807 04:15:31.477128 19204 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:15:31.480285 11164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:31.481633 11164 log.cc:824] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:31.479485 10980 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b58ff9a83dc04660b724e20b29612de6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766
I20260807 04:15:31.484370 11006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 4
I20260807 04:15:31.484722 11164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: No bootstrap required, opened a new log
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.62@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
I20260807 04:15:31.494649 11164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.495007 11164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ac38e8d69c4dbb986993556dc695e5, State: Initialized, Role: FOLLOWER
I20260807 04:15:31.495568 11164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.496934 11173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.497385 11173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:31.498119 11164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:31.501291 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.485033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56754)
0807 04:15:31.485252 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.485277 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:31.485396 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:31.485747 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:31.485752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:31.486263 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:31.486507 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.486520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.487960 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.487966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.489482 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.489497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.490611 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.490618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.490838 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.491598 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:31.491622 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:31.494412 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:31.498560 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.498566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.498570 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.498930 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.499327 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.499332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.499336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.499478 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.500175 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:31.500183 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:31.500368 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 04:15:31.500570 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 04:15:31.500884 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 04:15:31.501542 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.485196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.18.193.60:43501)
0807 04:15:31.485424 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.485432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.485454 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:15:31.485525 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.485530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.485601 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.485691 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.486658 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.487814 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.489630 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.490448 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.490573 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.494620 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.494627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.494631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.494670 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.498416 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.499057 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.499063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.499066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.499207 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.499589 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.499596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.499599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.500048 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.500235 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.500724 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.501019 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:15:31.505921 11006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6: Bootstrap starting.
I20260807 04:15:31.510308 11006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:31.511711 11006 log.cc:824] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6: Log is configured to *not* fsync() on all Append() calls
W20260807 04:15:31.514554 11184 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:15:31.514671 11184 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:15:31.519467 11006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6: No bootstrap required, opened a new log
I20260807 04:15:31.512219 11092 master_runner.cc:186] Error getting master registration for 127.18.193.62:37367: OK, instance_id { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" instance_seqno: 1786076130341399 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:15:31.521433 11092 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:15:31.526994 11006 raft_consensus.cc:348] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.527415 11006 raft_consensus.cc:729] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58ff9a83dc04660b724e20b29612de6, State: Initialized, Role: FOLLOWER
I20260807 04:15:31.527907 11006 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.528985 11187 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.529436 11187 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:31.530187 11006 sys_catalog.cc:564] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:31.530447 10980 raft_consensus.cc:482] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 04:15:31.530642 10980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:31.534355 10980 raft_consensus.cc:504] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.536868 11138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b58ff9a83dc04660b724e20b29612de6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "13ac38e8d69c4dbb986993556dc695e5"
I20260807 04:15:31.537374 11138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:31.540947 11138 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b58ff9a83dc04660b724e20b29612de6 in term 1.
I20260807 04:15:31.541780 10953 leader_election.cc:302] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ac38e8d69c4dbb986993556dc695e5, b58ff9a83dc04660b724e20b29612de6; no voters:
I20260807 04:15:31.542363 10980 leader_election.cc:288] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [CANDIDATE]: Term 1 election: Requested vote from peers d7b9011430714346aa668cce9a7e447d (127.18.193.61:37179), 13ac38e8d69c4dbb986993556dc695e5 (127.18.193.60:43501)
I20260807 04:15:31.542289 11187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:15:31.558172 11187 raft_consensus.cc:686] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [term 1 LEADER]: Becoming Leader. State: Replica: b58ff9a83dc04660b724e20b29612de6, State: Running, Role: LEADER
I20260807 04:15:31.558856 11187 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.574091 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.546608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40892 (local address 127.18.193.61:37179)
0807 04:15:31.546807 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.546814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.547866 (+ 1052us) server_negotiation.cc:407] Connection header received
0807 04:15:31.548297 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.548304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.548392 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.548470 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.552332 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.553474 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.556603 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.557268 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.557395 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.564578 (+ 7183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.564585 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.564590 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.564632 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.568417 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.571579 (+ 3162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.571587 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.571592 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.571751 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.572281 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.572286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.572289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.572693 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.572933 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.573494 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.573725 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 04:15:31.577520 11207 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 04:15:31.577195 11198 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58ff9a83dc04660b724e20b29612de6. Latest consensus state: current_term: 1 leader_uuid: "b58ff9a83dc04660b724e20b29612de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.577728 11198 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: This master's current role is: LEADER
W20260807 04:15:31.578986 10955 tablet.cc:2369] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:15:31.578526 11209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:15:31.589897 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } attempt: 5
I20260807 04:15:31.590900 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.546316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.18.193.62:37367)
0807 04:15:31.546529 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.546535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.560310 (+ 13775us) server_negotiation.cc:407] Connection header received
0807 04:15:31.560606 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.560612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.560708 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.560778 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.563334 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.564782 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.568110 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.569069 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.569195 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.579931 (+ 10736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.579940 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.579947 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.579988 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.583794 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.585116 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.585123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.585130 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.585310 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.588008 (+ 2698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.588014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.588018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.588398 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.589001 (+ 603us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.590236 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.590506 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:15:31.581750 10933 master_runner.cc:186] Error getting master registration for 127.18.193.61:37179: OK, instance_id { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" instance_seqno: 1786076130858573 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:15:31.594040 10933 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:15:31.597506 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.546882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.18.193.60:43501)
0807 04:15:31.547051 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.547059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.557850 (+ 10791us) server_negotiation.cc:407] Connection header received
0807 04:15:31.558301 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.558306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.558403 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.558478 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.569499 (+ 11021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.570664 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.579853 (+ 9189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.580610 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.580755 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.587724 (+ 6969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.587731 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.587735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.587776 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.591462 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.594313 (+ 2851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.594318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.594321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.594498 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.595228 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.595232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.595235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.595661 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.595922 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.596830 (+ 908us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.597077 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
I20260807 04:15:31.603050 11209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 04:15:31 dist-test-slave-8n49 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076130, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.193.61@KRBTEST.COM for kudu/127.18.193.60@KRBTEST.COM
I20260807 04:15:31.624006 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.591856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.193.60:43501 (local address 127.0.0.1:56772)
0807 04:15:31.592123 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 04:15:31.592141 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:15:31.592244 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:15:31.593765 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:15:31.593773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:15:31.594313 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:15:31.594531 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.594547 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.597708 (+ 3161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.597715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.599122 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:15:31.599138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.602628 (+ 3490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.602635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:15:31.602906 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.603675 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:15:31.603704 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:15:31.606562 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:15:31.620021 (+ 13459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.620027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.620031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.620702 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.621548 (+ 846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:15:31.621553 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:15:31.621557 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:15:31.621742 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:15:31.622922 (+ 1180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:15:31.622928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:15:31.623105 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 04:15:31.623247 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 04:15:31.623598 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260807 04:15:31.629844 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.592182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.18.193.60:43501)
0807 04:15:31.593198 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.593205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.593254 (+ 49us) server_negotiation.cc:407] Connection header received
0807 04:15:31.593345 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.593352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.593514 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.594715 (+ 1201us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.595217 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.597463 (+ 2246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.601021 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.602379 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.606950 (+ 4571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.607361 (+ 411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:15:31.607369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:15:31.607402 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:15:31.609825 (+ 2423us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:15:31.619751 (+ 9926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.621006 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.621012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.621016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.621344 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:15:31.622015 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:15:31.622020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:15:31.622023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:15:31.622712 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:15:31.628301 (+ 5589us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.628590 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.629045 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":611,"threads_started":1}
W20260807 04:15:31.636806 11116 tablet.cc:2369] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:15:31.636510 11085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d: Bootstrap starting.
I20260807 04:15:31.655320 11085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d: Neither blocks nor log segments found. Creating new log.
I20260807 04:15:31.657008 11085 log.cc:824] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:15:31.657172 11138 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [term 1 FOLLOWER]: Refusing update from remote peer b58ff9a83dc04660b724e20b29612de6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:15:31.658279 11198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:15:31.671475 11085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d: No bootstrap required, opened a new log
I20260807 04:15:31.683512 11173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58ff9a83dc04660b724e20b29612de6. Latest consensus state: current_term: 1 leader_uuid: "b58ff9a83dc04660b724e20b29612de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.683856 11173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:31.688868 11085 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.689323 11085 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b9011430714346aa668cce9a7e447d, State: Initialized, Role: FOLLOWER
I20260807 04:15:31.689848 11085 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } }
I20260807 04:15:31.690747 11198 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b58ff9a83dc04660b724e20b29612de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.691057 11198 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:31.692152 11214 mvcc.cc:205] Tried to move back new op lower bound from 7315767835256303616 to 7315767834871664640. Current Snapshot: MvccSnapshot[applied={T|T < 7315767835256303616}]
I20260807 04:15:31.693192 11209 catalog_manager.cc:1347] Generated new cluster ID: 7d23b91c4e2b44cdbc7b846ff748d352
I20260807 04:15:31.693306 11209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:15:31.702354 11085 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:15:31.702433 11059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b58ff9a83dc04660b724e20b29612de6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d7b9011430714346aa668cce9a7e447d"
I20260807 04:15:31.702845 11057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:15:31.699368 11218 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.703754 11218 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:31.706874 11057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [term 1 FOLLOWER]: Refusing update from remote peer b58ff9a83dc04660b724e20b29612de6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:15:31.707960 11187 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:15:31.711807 11059 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b58ff9a83dc04660b724e20b29612de6 in term 1.
I20260807 04:15:31.726137 11173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b58ff9a83dc04660b724e20b29612de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.726449 11173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:31.745612 11209 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:15:31.747551 11209 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:15:31.761847 11226 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b58ff9a83dc04660b724e20b29612de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.762043 11198 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b58ff9a83dc04660b724e20b29612de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.762344 11198 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6 [sys.catalog]: This master's current role is: LEADER
I20260807 04:15:31.762269 11218 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58ff9a83dc04660b724e20b29612de6. Latest consensus state: current_term: 1 leader_uuid: "b58ff9a83dc04660b724e20b29612de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b58ff9a83dc04660b724e20b29612de6" member_type: VOTER last_known_addr { host: "127.18.193.62" port: 37367 } } peers { permanent_uuid: "d7b9011430714346aa668cce9a7e447d" member_type: VOTER last_known_addr { host: "127.18.193.61" port: 37179 } } peers { permanent_uuid: "13ac38e8d69c4dbb986993556dc695e5" member_type: VOTER last_known_addr { host: "127.18.193.60" port: 43501 } } }
I20260807 04:15:31.762524 11226 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:31.762578 11218 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:15:31.771603 11209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b58ff9a83dc04660b724e20b29612de6: Generated new TSK 0
I20260807 04:15:31.772979 11209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:15:31.791385 11234 catalog_manager.cc:1259] Loaded cluster ID: 7d23b91c4e2b44cdbc7b846ff748d352
I20260807 04:15:31.791507 11234 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d: loading cluster ID for follower catalog manager: success
I20260807 04:15:31.795188 11234 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d: acquiring CA information for follower catalog manager: success
I20260807 04:15:31.797578 11234 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d7b9011430714346aa668cce9a7e447d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 04:15:31.925414 11171 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:31.912309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.18.193.61:37179)
0807 04:15:31.912532 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.912539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.923369 (+ 10830us) server_negotiation.cc:407] Connection header received
0807 04:15:31.923758 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.923764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.923871 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.923954 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.925116 (+ 1162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40902: BlockingRecv error: recv got EOF from 127.0.0.1:40902 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 04:15:31.927459 11170 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:31.908032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.18.193.62:37367)
0807 04:15:31.908241 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.908251 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.922592 (+ 14341us) server_negotiation.cc:407] Connection header received
0807 04:15:31.922815 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.922823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.922914 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.923040 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.927175 (+ 4135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38774: BlockingRecv error: recv got EOF from 127.0.0.1:38774 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 04:15:31.929570 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:31.918393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.18.193.60:43501)
0807 04:15:31.918578 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.918585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.926159 (+ 7574us) server_negotiation.cc:407] Connection header received
0807 04:15:31.927957 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.927963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.928069 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.928146 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:31.929337 (+ 1191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56776: BlockingRecv error: recv got EOF from 127.0.0.1:56776 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:15:31.978314 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:31.961793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40908 (local address 127.18.193.61:37179)
0807 04:15:31.962003 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:31.962009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:31.969559 (+ 7550us) server_negotiation.cc:407] Connection header received
0807 04:15:31.970156 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:31.970163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:31.970419 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:31.970570 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:31.971700 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.972897 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.975023 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:31.975728 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:31.975834 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:31.976684 (+ 850us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:31.977406 (+ 722us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:31.977551 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:31.977824 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 04:15:31.977942 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 04:15:32.021790 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.018593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.18.193.60:43501)
0807 04:15:32.018825 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.018836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.019393 (+ 557us) server_negotiation.cc:407] Connection header received
0807 04:15:32.019761 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.019768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.019908 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.020029 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.021464 (+ 1435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56784: BlockingRecv error: recv got EOF from 127.0.0.1:56784 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:15:32.075693 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.060251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.18.193.61:37179)
0807 04:15:32.060458 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.060464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.068088 (+ 7624us) server_negotiation.cc:407] Connection header received
0807 04:15:32.068699 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.068708 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.068807 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.068887 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.069659 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.070807 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.073032 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.073819 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.073997 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.074676 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.074908 (+ 232us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.074998 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.075255 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.075354 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 04:15:32.120385 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.110943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.18.193.60:43501)
0807 04:15:32.111173 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.111180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.118109 (+ 6929us) server_negotiation.cc:407] Connection header received
0807 04:15:32.118672 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.118678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.118788 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.118872 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.120095 (+ 1223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56786: BlockingRecv error: recv got EOF from 127.0.0.1:56786 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 04:15:32.172498 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.157155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.18.193.61:37179)
0807 04:15:32.157398 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.157406 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.165345 (+ 7939us) server_negotiation.cc:407] Connection header received
0807 04:15:32.165811 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.165818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.165931 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.166013 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.166776 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.167945 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.170052 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.170696 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.170808 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.171533 (+ 725us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.171717 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.171788 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.172062 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.172154 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 04:15:32.206111 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.203596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.18.193.60:43501)
0807 04:15:32.203808 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.203815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.204378 (+ 563us) server_negotiation.cc:407] Connection header received
0807 04:15:32.204615 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.204621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.204734 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.204872 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.205853 (+ 981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56796: BlockingRecv error: recv got EOF from 127.0.0.1:56796 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:15:32.221339 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.213966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40930 (local address 127.18.193.61:37179)
0807 04:15:32.214249 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.214256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.214616 (+ 360us) server_negotiation.cc:407] Connection header received
0807 04:15:32.214840 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.214847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.214946 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.215061 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.215576 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.216868 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.218877 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.219596 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.219734 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.220332 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.220482 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.220639 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.220868 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.220984 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260807 04:15:32.232864 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.230462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56808 (local address 127.18.193.60:43501)
0807 04:15:32.230752 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.230761 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.231043 (+ 282us) server_negotiation.cc:407] Connection header received
0807 04:15:32.231257 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.231263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.231527 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.231634 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.232591 (+ 957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56808: BlockingRecv error: recv got EOF from 127.0.0.1:56808 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
I20260807 04:15:32.252400 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.244970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.18.193.61:37179)
0807 04:15:32.245169 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.245175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.245743 (+ 568us) server_negotiation.cc:407] Connection header received
0807 04:15:32.245969 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.245976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.246097 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.246200 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.246749 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.248017 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.250091 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.250803 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.250964 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.251440 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.251599 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.251695 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.251880 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.251979 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 04:15:32.262171 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.259722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.18.193.60:43501)
0807 04:15:32.259926 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.259933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.260527 (+ 594us) server_negotiation.cc:407] Connection header received
0807 04:15:32.260770 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.260776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.260875 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.260989 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.261927 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56812: BlockingRecv error: recv got EOF from 127.0.0.1:56812 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:15:32.287909 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.281891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40942 (local address 127.18.193.61:37179)
0807 04:15:32.282127 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.282133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.282477 (+ 344us) server_negotiation.cc:407] Connection header received
0807 04:15:32.282675 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.282679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.282745 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.282825 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.283314 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.284096 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.286075 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.286508 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.286605 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.287154 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.287261 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.287334 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.287525 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.287593 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 04:15:32.296446 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.294326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.18.193.60:43501)
0807 04:15:32.294500 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.294504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.294854 (+ 350us) server_negotiation.cc:407] Connection header received
0807 04:15:32.295058 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.295062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.295140 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.295219 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.296177 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56828: BlockingRecv error: recv got EOF from 127.0.0.1:56828 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 04:15:32.339219 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.332753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.18.193.61:37179)
0807 04:15:32.332985 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.332991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.333541 (+ 550us) server_negotiation.cc:407] Connection header received
0807 04:15:32.333754 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.333760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.333845 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.333951 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.334457 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.335634 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.337105 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.337573 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.337725 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.338219 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.338381 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.338490 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.338682 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.338809 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 04:15:32.353883 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.346491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.18.193.60:43501)
0807 04:15:32.346728 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.346735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.352121 (+ 5386us) server_negotiation.cc:407] Connection header received
0807 04:15:32.352346 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.352354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.352495 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.352638 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.353660 (+ 1022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56834: BlockingRecv error: recv got EOF from 127.0.0.1:56834 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:15:32.427791 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.422375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40958 (local address 127.18.193.61:37179)
0807 04:15:32.422649 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.422657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.422974 (+ 317us) server_negotiation.cc:407] Connection header received
0807 04:15:32.423232 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.423236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.423307 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.423417 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.423855 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.424671 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.426019 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.426453 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.426614 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.427036 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.427159 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.427262 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.427415 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.427497 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 04:15:32.435557 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:15:32.433879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.18.193.60:43501)
0807 04:15:32.434066 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.434071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.434441 (+ 370us) server_negotiation.cc:407] Connection header received
0807 04:15:32.434597 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.434600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.434672 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.434763 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:15:32.435392 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56850: BlockingRecv error: recv got EOF from 127.0.0.1:56850 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:15:32.516816 11184 catalog_manager.cc:1259] Loaded cluster ID: 7d23b91c4e2b44cdbc7b846ff748d352
I20260807 04:15:32.516971 11184 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: loading cluster ID for follower catalog manager: success
I20260807 04:15:32.519549 11184 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: acquiring CA information for follower catalog manager: success
I20260807 04:15:32.521631 11184 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 13ac38e8d69c4dbb986993556dc695e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:15:32.573320 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.568049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.18.193.61:37179)
0807 04:15:32.568324 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.568331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.568582 (+ 251us) server_negotiation.cc:407] Connection header received
0807 04:15:32.568850 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.568855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.568925 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.569035 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.569461 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.570181 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.571548 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.571997 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.572140 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.572595 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.572697 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.572800 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.572958 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.573039 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 04:15:32.585654 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.579882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.18.193.60:43501)
0807 04:15:32.580077 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.580086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.580416 (+ 330us) server_negotiation.cc:407] Connection header received
0807 04:15:32.580663 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.580667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.580824 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.580949 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.581377 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.582091 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.583376 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.583808 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.583952 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.584446 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.585051 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.585213 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.585272 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.585355 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 04:15:32.603334 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.591775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.18.193.60:43501)
0807 04:15:32.591988 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.591996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.597300 (+ 5304us) server_negotiation.cc:407] Connection header received
0807 04:15:32.597489 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.597492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.597556 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.597660 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.598242 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.599141 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.601282 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.601861 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.601997 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.602492 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.602634 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.602717 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.602919 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.603002 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:15:32.603524 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.592176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.18.193.61:37179)
0807 04:15:32.592437 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.592447 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.597025 (+ 4578us) server_negotiation.cc:407] Connection header received
0807 04:15:32.597949 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.597955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.598051 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.598117 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.598818 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.599811 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.601519 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.602152 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.602304 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.602721 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.602876 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.603000 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.603130 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.603208 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:15:32.604912 11345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:15:32.591839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.18.193.62:37367)
0807 04:15:32.595711 (+ 3872us) server_negotiation.cc:206] Beginning negotiation
0807 04:15:32.595720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:15:32.596215 (+ 495us) server_negotiation.cc:407] Connection header received
0807 04:15:32.596579 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:15:32.596587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:15:32.596865 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:15:32.597059 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:15:32.597671 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.598854 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.601148 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:15:32.601708 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:15:32.601843 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:15:32.602598 (+ 755us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:15:32.603111 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:15:32.603211 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:15:32.603884 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 04:15:32.603987 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3708,"thread_start_us":3562,"threads_started":1}
I20260807 04:15:32.609877 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 10933
I20260807 04:15:32.611007 11001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:32.723819 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 10933
I20260807 04:15:32.747936 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 11014
I20260807 04:15:32.749166 11080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:32.858566 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 11014
I20260807 04:15:32.881119 19204 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5Tobyc/build/asan/bin/kudu pid 11092
I20260807 04:15:32.882385 11159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:15:32.991199 19204 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5Tobyc/build/asan/bin/kudu with pid 11092
2026-08-07T04:15:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4566 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9664 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (383518 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST